{shinyValidator}: toward a new audit process for Shiny apps quality through CI/CD

AppsilonConf 2022 Apr 29

David Granjon

Senior Expert Data Scientist

apps may look amazing ...

{Shiny} is about 10 years old 😲😲😲.

apps may look amazing ...

{Shiny} is about 10 years old 😲😲😲.

Today, many tools can make your apps shining...

Clothes don't make the man

Yet ... too many unexplained syndromes:

Clothes don't make the man

Yet ... too many unexplained syndromes:

The well known "It worked for me ... but not for you" 🤷‍♀️
The famous "It was fine yesterday" 😎
And many others...

Clothes don't make the man

Your app may be as beautiful and as cool as you want, it is useless if it does not start/run.

Most Shiny apps are "just" prototypes

Why is my app not starting/running ❓

Most Shiny apps are "just" prototypes

Why is my app not starting/running ❓

Possible answers:

Quickly developed, no planning 😱
No code quality management 😱
No tests 😱
...

Most Shiny apps are "just" prototypes

Why is my app not starting/running ❓

Possible answers:

Quickly developed, no planning 😱
No code quality management 😱
No tests 😱
...

This must stop 😈!

About software validation

Check that a product is conform to initial requirements (Software engineering).

About software validation

Check that a product is conform to initial requirements (Software engineering).

For Shiny apps:

Reliability: is the app doing what it is intended to do?
Stability: how often does it crash?
Performance/Availability: is the app fast enough to handle multiple concurrent users?

About software validation

Check that a product is conform to initial requirements (Software engineering).

For Shiny apps:

Reliability: is the app doing what it is intended to do?
Stability: how often does it crash?
Performance/Availability: is the app fast enough to handle multiple concurrent users?

In practice, a few apps meet all these requirements 😈. How do we transition from prototypes to production-ready tools ❓

1. Production Ready Shiny Apps

Consolidate Shiny apps projects

Easier checking, linting, documentation and testing.
Just ... easier. 😀

Fix package versions.
Increased reproducibility.

Unit tests: test business logic.
Server testing: test how Shiny modules or pieces work together (with reactivity).
UI testing: test UI components, snapshots, headless-testing ({shinytest2}).

Are there bottlenecks?

Load testing: How does the app behave with 10 simultaneous user? {shinyloadtest}.
Profiling: What part of my app is slow?{profvis}.
Reactivity: Are there any reactivity issues? .

Stop ... I am lost ...

There are just so many tools! How to use them properly?
Is there a way to automate all of this? I just don't have time ... 😞

3. Leveraging CI/CD10

What, why and how?

Continuous integration: automatically check new features. 🏥
Continuous deployment: automatically deploy content. ✉️
Running on a remote environment ☁️:
- Github Actions.
- Jenkins.
- Gitlab CI/CD.

Automated.
More reproducible (more os/R flavors available).
Time saver.
Less duplication.

Not easy 😢

Select DevOps platform (GitLab, GitHub, ...).
Add version control (git knowledge).
Build custom GitLab runner (optional).
Write CI/CD instructions (better support for GitHub).
Enjoy ...

Can't we make things easier❓

4. Welcome {shinyValidator}12

{shinyValidator}

Eases Shiny Apps validation:
- Provides plug and play CI/CD pipelines.
- Lints and checks the package code.
- Performs a Shiny app crash test.
- Displays outputs differences: plots, htmlwidgets.
- Measure performance: load-test, profiling and reactivity checkup.

All is integrated into a nice HTML report.

Requires a package, especially with {golem}.
Control R dependencies with {renv}.
Requires git + remote repository.
Have unit tests.

Install it:

remotes::install_github("Novartis/shinyValidator")

Run use_validator("github") (for Github Actions).
devtools::document().
Run renv::snapshot() to capture dependencies.
Commit, push and enjoy. 🚀

{shinyValidator}: audit app

audit_app() main package function:

audit_app(
  timeout = 5,
  headless_actions = NULL,
  workers = 5,
  scope = c("DMC", "POC"),
  output_validation = TRUE,
  coverage = TRUE,
  load_testing = TRUE,
  profile_code = TRUE,
  check_reactivity = TRUE,
  flow = TRUE
)

Live Demo

Because a demo is better than 30 slides.

Thanks 🙏🏻

Barret Schloerke (RStudio).
Peter Krusche (Novartis).
Lucas Widmer (Novartis).
Janice Branson (Novartis).
Technology & Innovation Team (Novartis): Aleks, Bo, Marie, Dave, David, Mustapha, Ardalan, Bill and Stefan.
Christophe Dervieux (RStudio).
Sebastien Rochette (ThinkR).
Romain Lesur (Insee).
Katerina Limpitsouni (unDraw, illustrations).

Follow me

Twitter.
Github.

{shinyValidator}: toward a new audit process for Shiny apps quality through CI/CD

AppsilonConf 2022 Apr 29

David Granjon

Senior Expert Data Scientist

apps may look amazing ...

{Shiny} is about 10 years old 😲😲😲.

apps may look amazing ...

{Shiny} is about 10 years old 😲😲😲.

Today, many tools can make your apps shining...

Clothes don't make the man

Yet ... too many unexplained syndromes:

Clothes don't make the man

Yet ... too many unexplained syndromes:

The well known "It worked for me ... but not for you" 🤷‍♀️
The famous "It was fine yesterday" 😎
And many others...

Clothes don't make the man

Your app may be as beautiful and as cool as you want, it is useless if it does not start/run.

Most Shiny apps are "just" prototypes

Why is my app not starting/running ❓

Most Shiny apps are "just" prototypes

Why is my app not starting/running ❓

Possible answers:

Quickly developed, no planning 😱
No code quality management 😱
No tests 😱
...

Most Shiny apps are "just" prototypes

Why is my app not starting/running ❓

Possible answers:

Quickly developed, no planning 😱
No code quality management 😱
No tests 😱
...

This must stop 😈!

About software validation

Check that a product is conform to initial requirements (Software engineering).

About software validation

Check that a product is conform to initial requirements (Software engineering).

For Shiny apps:

Reliability: is the app doing what it is intended to do?
Stability: how often does it crash?
Performance/Availability: is the app fast enough to handle multiple concurrent users?

About software validation

Check that a product is conform to initial requirements (Software engineering).

For Shiny apps:

Reliability: is the app doing what it is intended to do?
Stability: how often does it crash?
Performance/Availability: is the app fast enough to handle multiple concurrent users?

In practice, a few apps meet all these requirements 😈. How do we transition from prototypes to production-ready tools ❓

1. Production Ready Shiny Apps

Consolidate Shiny apps projects

Easier checking, linting, documentation and testing.
Just ... easier. 😀

Fix package versions.
Increased reproducibility.

Unit tests: test business logic.
Server testing: test how Shiny modules or pieces work together (with reactivity).
UI testing: test UI components, snapshots, headless-testing ({shinytest2}).

Are there bottlenecks?

Load testing: How does the app behave with 10 simultaneous user? {shinyloadtest}.
Profiling: What part of my app is slow?{profvis}.
Reactivity: Are there any reactivity issues? .

Stop ... I am lost ...

There are just so many tools! How to use them properly?
Is there a way to automate all of this? I just don't have time ... 😞

3. Leveraging CI/CD10

What, why and how?

Continuous integration: automatically check new features. 🏥
Continuous deployment: automatically deploy content. ✉️
Running on a remote environment ☁️:
- Github Actions.
- Jenkins.
- Gitlab CI/CD.

Automated.
More reproducible (more os/R flavors available).
Time saver.
Less duplication.

Not easy 😢

Select DevOps platform (GitLab, GitHub, ...).
Add version control (git knowledge).
Build custom GitLab runner (optional).
Write CI/CD instructions (better support for GitHub).
Enjoy ...

Can't we make things easier❓

4. Welcome {shinyValidator}12

{shinyValidator}

Eases Shiny Apps validation:
- Provides plug and play CI/CD pipelines.
- Lints and checks the package code.
- Performs a Shiny app crash test.
- Displays outputs differences: plots, htmlwidgets.
- Measure performance: load-test, profiling and reactivity checkup.

All is integrated into a nice HTML report.

Requires a package, especially with {golem}.
Control R dependencies with {renv}.
Requires git + remote repository.
Have unit tests.

Install it:

remotes::install_github("Novartis/shinyValidator")

Run use_validator("github") (for Github Actions).
devtools::document().
Run renv::snapshot() to capture dependencies.
Commit, push and enjoy. 🚀

{shinyValidator}: audit app

audit_app() main package function:

audit_app(
  timeout = 5,
  headless_actions = NULL,
  workers = 5,
  scope = c("DMC", "POC"),
  output_validation = TRUE,
  coverage = TRUE,
  load_testing = TRUE,
  profile_code = TRUE,
  check_reactivity = TRUE,
  flow = TRUE
)

Live Demo

Because a demo is better than 30 slides.

Thanks 🙏🏻

Barret Schloerke (RStudio).
Peter Krusche (Novartis).
Lucas Widmer (Novartis).
Janice Branson (Novartis).
Technology & Innovation Team (Novartis): Aleks, Bo, Marie, Dave, David, Mustapha, Ardalan, Bill and Stefan.
Christophe Dervieux (RStudio).
Sebastien Rochette (ThinkR).
Romain Lesur (Insee).
Katerina Limpitsouni (unDraw, illustrations).

Follow me

Twitter.
Github.

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help
o	Tile View: Overview of Slides
Alt + f	Fit Slides to Screen

{shinyValidator}: toward a new audit process for Shiny apps quality through CI/CD

AppsilonConf 2022 Apr 29

apps may look amazing ...

apps may look amazing ...

Clothes don't make the man

Clothes don't make the man

Clothes don't make the man

Most Shiny apps are "just" prototypes

Most Shiny apps are "just" prototypes

Most Shiny apps are "just" prototypes

About software validation

About software validation

About software validation

1. Production Ready Shiny Apps

Consolidate Shiny apps projects

Stop ... I am lost ...

3. Leveraging CI/CD

What, why and how?

4. Welcome {shinyValidator}

{shinyValidator}

{shinyValidator}: audit app

Live Demo

Thanks 🙏🏻

Follow me

apps may look amazing ...

Help

{shinyValidator}: toward a new audit process for Shiny apps quality through CI/CD

{shinyValidator}: toward a new audit process for Shiny apps quality through CI/CD

AppsilonConf 2022 Apr 29

apps may look amazing ...

apps may look amazing ...

Clothes don't make the man

Clothes don't make the man

Clothes don't make the man

Most Shiny apps are "just" prototypes

Most Shiny apps are "just" prototypes

Most Shiny apps are "just" prototypes

About software validation

About software validation

About software validation

1. Production Ready Shiny Apps

Consolidate Shiny apps projects

Stop ... I am lost ...

3. Leveraging CI/CD

What, why and how?

4. Welcome {shinyValidator}

{shinyValidator}

{shinyValidator}: audit app

Live Demo

Thanks 🙏🏻

Follow me