Skip to content

Commit

Permalink
Organised examples from README
Browse files Browse the repository at this point in the history
  • Loading branch information
@oyama
oyama committed Jun 11, 2024
1 parent 1bc28ba commit 0f86709
Show file tree
Hide file tree
Showing 2 changed files with 62 additions and 158 deletions.
26 changes: 23 additions & 3 deletions EXAMPLE.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Example of pico-vfs library

## Examples

| App | Description |
|-----------------|-------------------------------------------------------------------------|
| [hello](hello) | Hello filesystem world. |
Expand All @@ -10,11 +8,33 @@
| [usb\_msc\_logger](usb_msc_logger) | Data logger that mounts littlefs and FAT on flash memory and shares it with a PC via USB mass storage class.|


Examples include benchmark test firmware for combinations of heterogeneous block devices and heterogeneous file systems, and logger firmware that splits the on-board flash memory in two and uses it with different file systems.

The [pico-sdk](https://github.com/raspberrypi/pico-sdk) build environment is required to build the demonstration, see [Getting started with Raspberry Pi Pico](https://datasheets.raspberrypi.com/pico/getting-started-with-pico.pdf) to prepare the toolchain for your platform. This project contains a _git submodule_. When cloning, the `--recursive` option must be given or a separate `git submodule update` must be performed.

## Building sample code

Firmware can be built from the _CMake_ build directory of _pico-vfs_.

```bash
cd build/
mkdir build; cd build/
PICO_SDK_PATH=/path/to/pico-sdk cmake ..
make hello fs_init_example benchmark logger
```
If no SD card device is connected, the `-DWITHOUT_BLOCKDEVICE_SD` option can be specified to skip the SD card manipulation procedure from the demo and unit tests.

### Circuit Diagram

If an SD card is to be used, a separate circuit must be connected via SPI. As an example, the schematic using the [Adafruit MicroSD card breakout board+](https://www.adafruit.com/product/254) is as follows

![adafruit-microsd](https://github.com/oyama/pico-vfs/assets/27072/b96e8493-4f3f-4d44-964d-8ada61745dff)

The spi and pin used in the block device argument can be customised. The following pins are used in the demonstration.

| Pin | PCB pin | Usage | description |
|------|---------|----------|-------------------------|
| GP18 | 24 | SPI0 SCK | SPI clock |
| GP19 | 25 | SPI0 TX | SPI Master Out Slave In |
| GP16 | 21 | SPI0 RX | SPI Master In Slave Out |
| GP17 | 22 | SPI0 CSn | SPI Chip select |

194 changes: 39 additions & 155 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,25 +1,36 @@
# Thin virtual file system for Raspberry Pi Pico

__pico-vfs__ seamlessly integrates POSIX file APIs on the Raspberry Pi Pico, allowing users to interact with the file system using a familiar, standardized interface. pico-vfs offers the flexibility to freely combine various __block devices__ and __file systems__ to configure an integrated file system, and is easy to set up from the start.
pico-vfs is a thin virtual file system for the Raspberry Pi Pico that seamlessly integrates POSIX file APIs, allowing users to interact with the file system using a standard interface. This library provides the convenience of performing standard file operations common in Unix environments directly on a microcontroller. It offers flexibility in combining various block devices and file systems, allowing developers to create more creative solutions without being constrained by the environment.

## Key Features
## Features

- **POSIX API Support**: Utilize standard POSIX file operations such as `open`, `read`, `write`, and `close` to achieve intuitive file handling.
- **Modular Design**: Easily exchange underlying block devices and file systems without modifying application code, providing high flexibility.
- **Pre-configured File System**: Includes a default file system setup, reducing initial setup time and complexity.
- **Customizability**: Use the `pico_enable_filesystem` function in your project's CMake setup to configure and activate different file systems and block devices as needed.
- **POSIX and C Standard File API Support:** In addition to basic POSIX file operations like `open`, `read`, `write`, and `close`, higher-level file operation functions such as `fprintf`, `fwrite`, and `fread` are also available. This enables intuitive and efficient file handling familiar to C language programmers.
- **Modular Design:** The design of pico-vfs employs a multi-layer architecture consisting of a **block device** abstraction layer, a **file system** abstraction layer, and a **VFS** layer that integrates these into the POSIX API. This allows developers to easily select and swap underlying storage devices (e.g., flash memory, SD cards) and file systems (e.g., FAT, littlefs) without modifying application code.
- **Virtual File System:** Multiple different file systems can be mounted into a single namespace and operated with a consistent API. Programmers can seamlessly operate without being aware of the differences in storage media and file systems.
- **Pre-configured File System:** Includes a default file system setup to reduce initial setup time and complexity. Additionally, there is an `AUTO_INIT` feature that initializes the file system before the firmware's `main()` execution.

pico-vfs is designed with resource-constrained environments in mind, making it easy to customize and readily adaptable to specific project needs.
## Setup and Configuration

## Quick Start Guide
To add pico-vfs to your project, first place the pico-vfs source in your project directory using the following commands:

To use pico-vfs, add the file system enable instructions to your project's `CMakeLists.txt`:
```bash
git clone https://github.com/oyama/pico-vfs.git
cd pico-vfs
git submodule update --init
```

Next, add the following lines to your project's `CMakeLists.txt` to include pico-vfs in the build process:

```CMakeLists.txt
pico_enable_filesystem(${CMAKE_PROJECT_NAME})
add_subdirectory(pico-vfs)
```

Simply call the file system initialization function `fs_init()` from your program:
Then, add the `pico_enable_filesystem` function to `CMakeLists.txt` to enable the file system:

```CMakeLists.txt
pico_enable_filesystem(${CMAKE_PROJECT_NAME})
```
This sets up your project to use the pico-vfs file system functionality. From your program, call the file system initialization function `fs_init()` to start using it:

```c
#include <pico/stdlib.h>
Expand All @@ -30,170 +41,43 @@ int main(void) {
stdio_init_all();
fs_init();

FILE *fp = fopen("/HELLO.TXT", "w");
FILE *fp = fopen("HELLO.TXT", "w");
fprintf(fp, "Hello World!\n");
fclose(fp);
}
```
By default, 1.4MB of littlefs is mounted at `/`, and the Pico's onboard flash memory is used as a block device.
If you want to customize the storage size, specify the required storage size in `pico_enable_filesystem`:
```CMakeLists.txt
pico_enable_filesystem(${CMAKE_PROJECT_NAME} FS_SIZE 1441792)
```
pico-vfs can be further customised. The examples included in [examples/fs_inits](examples/fs_inits) show example configurations of different file system combinations. For example, a FAT file system on SD card, and littlefs on on-board flash memory can be mounted in a single namespace.

## Architecture overview

To provide a flexible and lightweight file system framework for embedded systems, pico-vfs consists of three layers: a __block device__ layer that abstracts storage devices, a __file system__ layer that abstracts file systems, and a __virtual file system__ layer that links these to the POSIX API.
## Usage Guide
### Block device layer
For detailed usage examples, refer to [EXAMPLE](EXAMPLE.md). For configuration examples involving various block devices and file systems, refer to [examples/fs\_inits](examples/fs_inits/). These include examples such as mounting a FAT file system on an SD card and littlefs on onboard flash memory into a single namespace.
A block device is an object implementing `blockdevice_t`, which includes callback functions and variables tailored to the block device, allowing different block devices to be operated with a consistent interface.

- [Flash Block Device](src/blockdevice/flash.c): Manages Raspberry Pi Pico onboard flash memory as a block device.
- [SD Card Block Device](src/blockdevice/sd.c): Manages SD/MMC cards as block devices via SPI interface.
- [Heap Block Device](src/blockdevice/heap.c): Manages heap memory for block device operations.
- [Loopback Block Device](src/blockdevice/loopback.c): Manages disk image file manipulation through a virtual block device.

### File system layer

The file system is an object implementing `filesystem_t`, which contains callback functions and variables based on the file system, enabling different file systems to be operated with a consistent interface.

- [FAT File System](src/filesystem/fat.c): Manages FAT filesystem operation via FatFs[^1].
- [littlefs File System](src/filesystem/littlefs.c): Manages operations for the littlefs[^2] filesystem.

### VFS layer

Block devices and file systems are integrated into the POSIX file API by the VFS layer. Users can perform operations like `open()`, `read()`, and `write()` as usual, and also use higher-level functions such as `fopen()`, `fprintf()`, and `fgets()`. The available POSIX and C standard file APIs are listed in [STANDARD.md](STANDARD.md). See [API.md](API.md) for file system management APIs not defined in POSIX.

- [Virtual File System](src/filesystem/vfs.c): Integrates block devices and file systems with POSIX file APIs, enabling standard file operations.

Furthermore, individual block devices and file systems are available as separate INTERFACE libraries, enabling the file system to be configured with minimal footprint.

## Examples

Examples include benchmark test firmware for combinations of heterogeneous block devices and heterogeneous file systems, and logger firmware that splits the on-board flash memory in two and uses it with different file systems.

The pico-sdk[^3] build environment is required to build the demonstration, see _Getting started with Raspberry Pi Pico_[^4] to prepare the toolchain for your platform. This project contains a _git submodule_. When cloning, the `--recursive` option must be given or a separate `git submodule update` must be performed.

```bash
git clone --recursive https://github.com/oyama/pico-vfs.git
cd pico-vfs
mkdir build; cd build
PICO_SDK_FETCH_FROM_GIT=1 cmake ..
make benchmark logger default_fs tests
```
The above examples specify the environment variable `PICO_SDK_FETCH_FROM_GIT` to download the pico-sdk from GitHub. If you want to specify a locally deployed pico-sdk, you should set it with the `PICO_SDK_PATH` environment variable.
The build generates the firmware `examples/benchmark/benchmark.uf2`, `examples/usb_msc_logger/logger.uf2` and `examples/default_fs/default_fs.uf2`. Both can be installed by simply dragging and dropping them onto a Raspberry Pi Pico running in BOOTSEL mode.
Additionally, a list of POSIX standard file APIs verified to work with pico-vfs is available in [STANDARD](STANDARD.md). Refer to this document to check which APIs are available. For management APIs not included in the POSIX standard, such as file system formatting and mounting, refer to [API](API.md).
If no SD card device is connected, the `WITHOUT_BLOCKDEVICE_SD` option can be specified to skip the SD card manipulation procedure from the demo and unit tests.

```bash
PICO_SDK_FETCH_FROM_GIT=1 cmake .. -DWITHOUT_BLOCKDEVICE_SD=YES
```

### Circuit Diagram

If an SD card is to be used, a separate circuit must be connected via SPI. As an example, the schematic using the Adafruit MicroSD card breakout board+[^5] is as follows

![adafruit-microsd](https://github.com/oyama/pico-vfs/assets/27072/b96e8493-4f3f-4d44-964d-8ada61745dff)

The spi and pin used in the block device argument can be customised. The following pins are used in the demonstration.

| Pin | PCB pin | Usage | description |
|------|---------|----------|-------------------------|
| GP18 | 24 | SPI0 SCK | SPI clock |
| GP19 | 25 | SPI0 TX | SPI Master Out Slave In |
| GP16 | 21 | SPI0 RX | SPI Master In Slave Out |
| GP17 | 22 | SPI0 CSn | SPI Chip select |

## Integration into project

Add the pico-vfs repository to your project by `git submodule` or simply `git clone`. Since pico-vfs contains submodules, recommended with `git submodule update --init --recursive`:

```bash
git submodule add https://github.com/oyama/pico-vfs.git
git submodule update --init --recursive
```

Add `add_subdirectory` to `CMakeLists.txt` so that they can be built together:
```CMakeLists.txt
add_subdirectory(pico-vfs)
pico_enable_filesystem(${CMAKE_PROJECT_NAME})
```

In this example, a minimal number of components are configured by default.

For instance, a configuration where the onboard flash memory is set up with littlefs and combined with the FAT file system on an SD card is as follows:

```CMakeLists.txt
pico_enable_filesystem(${CMAKE_PROJECT_NAME} FS_INIT my_fs_init.c)
target_link_libraries(${CMAKE_PROJECT_NAME} PRIVATE
blockdevice_flash
blockdevice_sd
filesystem_littlefs
filesystem_fat
filesystem_vfs
```
You can add the `fs_init()` function to your project to freely layout the file system.

```my_fs_init.c
#include <hardware/clocks.h>
#include <hardware/flash.h>
#include "blockdevice/flash.h"
#include "blockdevice/sd.h"
#include "filesystem/fat.h"
#include "filesystem/littlefs.h"
#include "filesystem/vfs.h"

bool fs_init(void) {
blockdevice_t *flash = blockdevice_flash_create(PICO_FLASH_SIZE_BYTES - DEFAULT_FS_SIZE, 0);
blockdevice_t *sd = blockdevice_sd_create(spi0,
PICO_DEFAULT_SPI_TX_PIN,
PICO_DEFAULT_SPI_RX_PIN,
PICO_DEFAULT_SPI_SCK_PIN,
PICO_DEFAULT_SPI_CSN_PIN,
10 * MHZ,
false);
filesystem_t *lfs = filesystem_littlefs_create(500, 16);
filesystem_t *fat = filesystem_fat_create();

fs_mount("/", lfs, flash);
fs_mount("/sd", fat, sd);
return true;
}
```
## Limitations
Of course, a more bare-metal use, where only block devices are utilized, is also possible.
For detailed limitations, refer to [LIMITATION](LIMITATION.md). This document provides detailed explanations of potential limitations in specific scenarios and configurations.
## Limitations
## License
For detailed information on the limitations and considerations when using pico-vfs, especially with regard to multi-core operations, please see [LIMITATION.md](LIMITATION.md).
This project is licensed under the 3-Clause BSD License. For details, see the [LICENSE](LICENSE.md) file.
## Related Projects and Inspirations
There are multiple ways to add filesystems to the pico-sdk environment. Firstly, FatFs[^1] and littlefs[^2] are popular file system implementations. These filesystem implementations require writing drivers for the block devices used. They also each have their own Unix-like API, but with a distinctive dialect.
While there are several solutions that cover the problem of writing drivers for the Raspberry Pi Pico, the carlk3[^6] implementation is probably the most popular. Especially, it includes DMA support to reduce CPU load and support for even faster SDIO[^7]. This would be the first choice for projects using SD cards and the FAT file system with pico-sdk.
While there are several solutions that cover the problem of writing drivers for the Raspberry Pi Pico, the carlk3[^3] implementation is probably the most popular. Especially, it includes DMA support to reduce CPU load and support for even faster SDIO[^4]. This would be the first choice for projects using SD cards and the FAT file system with pico-sdk.
Among multi-filesystem implementations, Memotech-Bill[^8] implementation provides standard I/O support for pico-sdk using the newlib[^9] hook. The littlefs file system for on-board flash and FatFs for SD cards can be operated as an integrated file system. It is an ambitious project that goes beyond files and integrates character devices such as TTYs and UARTs.
Among multi-filesystem implementations, Memotech-Bill[^5] implementation provides standard I/O support for pico-sdk using the Newlib[^6] hook. The littlefs file system for on-board flash and FatFs for SD cards can be operated as an integrated file system. It is an ambitious project that goes beyond files and integrates character devices such as TTYs and UARTs.
While referring to these existing projects, _pico-vfs_ aims to make the implementation of drivers and file systems for block devices separate and interchangeable, similar to MicroPython's VFS[^10] and ARM Mbed OS's Storage[^11].
While referring to these existing projects, _pico-vfs_ aims to make the implementation of drivers and file systems for block devices separate and interchangeable, similar to MicroPython's VFS[^7] and ARM Mbed OS's Storage[^8].
## References
[^1]: [Generic FAT Filesystem Module](http://elm-chan.org/fsw/ff/)
[^2]: [littlefs](https://github.com/littlefs-project/littlefs)
[^3]: [pico-sdk](https://github.com/raspberrypi/pico-sdk)
[^4]: [Getting started with Raspberry Pi Pico](https://datasheets.raspberrypi.com/pico/getting-started-with-pico.pdf)
[^5]: [adafruit MicroSD card breakout board+](https://www.adafruit.com/product/254)
[^6]: [C/C++ Library for SD Cards on the Pico](https://github.com/carlk3/no-OS-FatFS-SD-SDIO-SPI-RPi-Pico)
[^7]: [SDIO/iSDIO](https://www.sdcard.org/developers/sd-standard-overview/sdio-isdio/)
[^8]: [Standard File Input / Output for the Pico SDK](https://github.com/Memotech-Bill/pico-filesystem)
[^9]: [Newlib is a C library intended for use on embedded systems](https://www.sourceware.org/newlib/)
[^10]: [MicroPython Working with filesystems](https://docs.micropython.org/en/latest/reference/filesystem.html)
[^11]: [ARM Mbed OS - Data storage concepts](https://os.mbed.com/docs/mbed-os/v6.16/apis/data-storage-concepts.html)
[^3]: [C/C++ Library for SD Cards on the Pico](https://github.com/carlk3/no-OS-FatFS-SD-SDIO-SPI-RPi-Pico)
[^4]: [SDIO/iSDIO](https://www.sdcard.org/developers/sd-standard-overview/sdio-isdio/)
[^5]: [Standard File Input / Output for the Pico SDK](https://github.com/Memotech-Bill/pico-filesystem)
[^6]: [Newlib is a C library intended for use on embedded systems](https://www.sourceware.org/newlib/)
[^7]: [MicroPython Working with filesystems](https://docs.micropython.org/en/latest/reference/filesystem.html)
[^8]: [ARM Mbed OS - Data storage concepts](https://os.mbed.com/docs/mbed-os/v6.16/apis/data-storage-concepts.html)

0 comments on commit 0f86709

Please sign in to comment.