Skip to content

Commit

Permalink
Update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
soulgalore committed Dec 15, 2023
1 parent 4261c3f commit cfb289d
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 57 deletions.
6 changes: 3 additions & 3 deletions docs/documentation/sitespeed.io/compare/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ twitterdescription: Use Mann Whitney U or Wilcox statistical methods to know if
---
[Documentation]({{site.baseurl}}/documentation/sitespeed.io/) / Compare

# Compare
# Compare - Statistical Methods for Regression Analysis
{:.no_toc}

* Let's place the TOC here
{:toc}

In the intricate world of web performance analysis, detecting performance regressions can be difficult. Subtle changes in your website code or infrastructure can lead to significant variations in performance, often hidden beneath layers of data. This is where the Mann Whitney U and Wilcoxon signed-rank tests, incorporated into sitespeed.io's compare plugin, shine as powerful statistical tools.
Sitespeed.io utilizes Mann Whitney U and Wilcoxon tests for detecting performance regressions.

## Why Mann Whitney U and Wilcoxon for Web Performance?

Expand Down Expand Up @@ -87,7 +87,7 @@ The columns:
6. **Current Median**: The median value for the current test run for each metric.
7. **Baseline Std Dev**: Standard deviation for the baseline test run for each metric. This measures the amount of variation from the average.
8. **Current Std Dev**: Standard deviation for the current test run for each metric.
9. **Significant Change?**: This column indicates whether the change between the baseline and current test runs is statistically significant for each metric.
9. **Significant Change?**: This column indicates whether the change between the baseline and current test runs is statistically significant for each metric. If the change is statistically significant then we use [Cliffs Delta](https://en.wikipedia.org/wiki/Effect_size#Effect_size_for_ordinal_data) to know if the change is small, medium or large.


And the compare graphs will look like this for every metric:
Expand Down
23 changes: 11 additions & 12 deletions docs/documentation/sitespeed.io/installation/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,15 @@ twitterdescription: Install sitespeed.io using npm, yarn or Docker.
* Lets place the TOC here
{:toc}

# Install
You can run sitespeed.io using our Docker containers or using NodeJS.
# Installation

You can install sitespeed.io using Docker or NodeJS.

## Docker
## Using Docker

We have [Docker images](https://hub.docker.com/r/sitespeedio/sitespeed.io/) with sitespeed.io, Chrome, Firefox, Edge, Xvfb and all the software needed for recording a video of the browser screen and analyse it to get Visual Metrics. It is super easy to use). Here's how to use the container with both Firefox & Chrome (install [Docker](https://docs.docker.com/install/) first).
Docker images include sitespeed.io, browsers (Chrome, Firefox, Edge), and tools for video recording and analysis.

### Mac & Linux
### Example command (Mac & Linux)

~~~bash
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io https://www.sitespeed.io -b firefox
Expand All @@ -38,18 +39,17 @@ C:\Users\Vicky> docker run --rm -v ${pwd}:/sitespeed.io sitespeedio/sitespeed.io

That will output the data from the run in the current directory. You can read more about running the containers [here](/documentation/sitespeed.io/docker/).

## Node JS
## Using Node JS

### Mac
To be able to record a video of the screen and analyse the video, you need a couple of extra software except sitespeed.io.
Requires additional software like FFmpeg, ImageMagick, and Python dependencies.

You need: [FFmpeg](https://ffmpeg.org), [ImageMagick 6](https://imagemagick.org/index.php) and [pillow](https://pillow.readthedocs.io/en/stable/).
### Installation steps for Apple Mac M1

Install on a fresh Apple Mac M1:

1. Install Homebrew [https://brew.sh](https://brew.sh)
1. Install Homebrew: `/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"`
2. Install latest NodeJS LTS (and npm). Either download it from [nodejs.org](https://nodejs.org/en/) or install using Homebrew:
`brew install node@16`
`brew install node@20`
3. Make sure you can install using *npm* without using sudo. Checkout [Sindre Sorhus guide](https://github.com/sindresorhus/guides/blob/master/npm-global-without-sudo.md).
4. Install ImageMagick 6
`brew install imagemagick@6`
Expand Down Expand Up @@ -80,7 +80,6 @@ After that you can also install the browsers that you need for your testing: [Ch

### Linux


If you are using Ubuntu you can use our prebuilt script. It will install all dependencies that you need to run sitespeed.io including latest Firefox and Chrome. Use it if you have a new machine or just setup a new cloud instance. It will also create a new user *sitespeedio* that you will use to run sitespeed.io. The script will ask for a new password for that user:

~~~bash
Expand Down
63 changes: 21 additions & 42 deletions docs/documentation/sitespeed.io/introduction/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,45 +16,24 @@ twitterdescription: Introduction for sitespeed.io.
* Let's place the TOC here
{:toc}

**Sitespeed.io is a *complete web performance tool* that helps you measure the performance of your website. What exactly does that mean?**

We think of a complete web performance tool as having three key capabilities:

- It should test web sites using real browsers, simulating real users connectivity and collect important user centric metrics like Speed Index and First Visual Render.
- It should analyse how your page is built and give feedback how you can make it faster for the end user.
- It should collect and keep data on how your pages are built so you can easily track changes.

**What is sitespeed.io good for?**

It is usually used in two different areas:

- Running in your continuous integration to find web performance regressions early: on commits or when you move code to your test environment
- Monitoring your performance in production, alerting on regressions.

To understand how sitespeed.io does these things, let's talk about how it works.

First a few key concepts:

- Sitespeed.io is built upon a couple of other Open Source tools in the sitespeed.io suite.
- [Browsertime](../../browsertime/) is the tool that drives the browser and collect metrics.
- [The Coach](../../coach/) knows how to build fast websites and analyse your page and give you feedback what you should change.
- Visual Metrics is metrics collected from a video recording of the browser screen.
- Everything in sitespeed.io is a [plugin](../plugins/) and they communicate by passing messages on a queue.

When you as user choose to test a URL, this is what happens on a high level:

1. sitespeed.io starts and initialises all configured plugins.
2. The URL is passed around the plugins through the queue.
1. Browsertime gets the URL and opens the browser.
2. It starts to record a video of the browser screen.
3. The browser accesses the URL.
4. When the page is finished, Browsertime takes a screenshot of the page.
5. It then runs some JavaScript to analyse the page (using Coach and Browsertime scripts).
6. It stops the video and closes the browser.
7. Analyses the video to get Visual Metrics like First Visual Change and Speed Index.
8. Browsertime passes all metrics and data on the queue so other plugins can use it.
3. The HTML/Graphite/InfluxDB plugin collects the metrics in queue.
4. When all URLs are tested, sitespeed sends a message telling plugins to summarise the metrics and then render it.
5. Plugins pickup the render message and the HTML plugin writes the HTML to disk.

Now it's time for you to [install and run sitespeed.io](../installation/).
Sitespeed.io is a complete web performance tool that helps you measure the performance of your website. It is designed to:

1. Test websites using real browsers, simulating real user connectivity.
2. Analyze your page's construction and provide feedback for speed optimization.
3. Collect and maintain data on page construction for easy tracking of changes.

**Use Cases**
- **Continuous Integration**: Detect web performance regressions early in the development cycle.
- **Production Monitoring**: Monitor performance in production and get alerted on regressions.
- **Web performance audit**: Run performance tests from your terminal.

**How it Works**
- Built on open source tools like Browsertime and The Coach.
- Uses a plugin-based architecture for flexibility.
- High-level process: Initialization → URL Testing → Metrics Collection → Summary and Rendering.

**Example Workflow**
1. Initialize sitespeed.io and its plugins.
2. Test a URL: Open in browser → Record video → Analyze page → Collect metrics.
3. Collect and summarize metrics using plugins like HTML/Graphite/InfluxDB.
4. Generate and store HTML reports.

0 comments on commit cfb289d

Please sign in to comment.