mcu
/
spiffs
mirror of https://github.com/pellepl/spiffs
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

added more documentation, error result for prohibited r|w, MIN MAX define

pull/7/head
Peter Andersson 11 years ago
parent
28a9c82c0d
commit
4d01fd24a0
8 changed files with 354 additions and 49 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 14
      README
  2. 35
      docs/INTEGRATION
  3. 230
      docs/TECH_SPEC
  4. 69
      src/default/spiffs_config.h
  5. 25
      src/spiffs.h
  6. 15
      src/spiffs_hydrogen.c
  7. 10
      src/spiffs_nucleus.h
  8. 3
      src/test/params_test.h

14
README
View File

@ -12,7 +12,7 @@ Love to hear feedback though!
* INTRODUCTION
Spiffs is a file system intended for SPI flash devices on embedded targets.
Spiffs is a file system intended for SPI NOR flash devices on embedded targets.
Spiffs is designed with following characteristics in mind:
- Small (embedded) targets, sparse RAM
@ -22,7 +22,8 @@ Spiffs is designed with following characteristics in mind:
- Zeroes can only be pulled to ones by erase
- Wear leveling
** Features
* FEATURES
What spiffs does:
- Posix-like api: open, close, read, write, seek, stat, etc
@ -32,6 +33,7 @@ What spiffs does:
SPI flash device
- Implements static wear leveling
- Built in file system consistency checks
- Specifically designed for low ram usage
What spiffs does not:
- Presently, spiffs does not support directories. It produces a flat
@ -40,9 +42,15 @@ What spiffs does not:
- It is not a realtime stack. One write operation might take much longer than
another.
- Poor scalability. Spiffs is intended for small memory devices - the normal
sizes for SPI flashes. Going beyond ~128MB is probably a bad idea.
sizes for SPI flashes. Going beyond ~128MB is probably a bad idea. This is
a side effect of the design goal to use as little ram as possible.
- Presently, it does not detect or handle bad blocks.
* MORE INFO
For integration, see the docs/INTEGRATION file.
For use and design, see the docs/TECH_SPEC file.
For testing and contributions, see the docs/IMPLEMENTING file.

35
docs/INTEGRATION
View File

@ -14,7 +14,7 @@ First and foremost, one must decide how to divide up the SPI flash for spiffs.
Having the datasheet for the actual SPI flash in hand will help. Spiffs can be
defined to use all or only parts of the SPI flash.
If following seems arcane, read the "DESIGN" chapter first.
If following seems arcane, read the "HOW TO CONFIG" chapter first.
- Decide the logical size of blocks. This must be a multiple of the biggest
physical SPI flash block size. To go safe, use the physical block size -
@ -85,16 +85,26 @@ The config struct must be initialized prior to mounting. One must always
define the SPI flash access functions:
spiffs_config.hal_read_f - pointing to the function reading the SPI flash
spiffs_config.hal_write_f - pointing to the function writing the SPI flash
spiffs_config.hal_erase_f - pointing to the function erasing the SPI flash
spiffs_config.hal_erase_f - pointing to the function erasing the SPI flash
Depending on the build config - if SPIFFS_SINGLETON is undefined - following
parameters must be defined:
spiffs_config.phys_size - the physical number of bytes accounted for
spiffs on the SPI flash
spiffs_config.phys_addr - the physical starting address on the SPI flash
TODO
spiffs_config.phys_erase_block - the physical size of the largest block/sector
on the SPI flash found within the spiffs
usage address space
spiffs_config.log_block_size -
spiffs_config.log_page_size -
** Build config
@ -107,6 +117,21 @@ be typedeffed.
spiffs_config.h: you also need to define a spiffs_config.h header. Example of
this is found in the default/ directory.
** RAM
Spiffs needs ram. It needs a working buffer being double the size of the
logical page size. It also needs at least one file descriptor. If cache is
enabled (highly recommended), it will also need a bunch of cache pages.
Say you have a logical page size of 256 bytes. You want to be able to have four
files open simultaneously, and you can give spiffs four cache pages. This sums
up to:
256*2 (work buffer) + 32*4 (file descriptors) + (256+32)*4 cache pages
i.e. 1792 bytes.
This is apart from call stack usage.
* QUICK AND DIRTY INTEGRATION EXAMPLE
@ -231,3 +256,7 @@ And, crossing fingers hard, you'll get the output:
--> Hello world <--
* HOW TO CONFIG
TODO

230
docs/TECH_SPEC
View File

@ -2,29 +2,235 @@
TODO
* DESIGN
** SPI flash devices
* SPIFFS DESIGN
Below is a small description of how SPI flashes work internally.
Spiffs is inspired by YAFFS. However, YAFFS is designed for NAND flashes, and
for bigger targets with much more ram. Nevertheless, many wise thoughts have
been borrowed from YAFFS when writing spiffs. Kudos!
The main complication writing spiffs was that it cannot be assumed the target
has a heap. Spiffs must go along only with the work ram buffer given to it.
This forces extra implementation on many areas of spiffs.
** SPI flash devices using NOR technology
Below is a small description of how SPI flashes work internally. This is to
give an understanding of the design choices made in spiffs.
SPI flash devices are physically divided in blocks. On some SPI flash devices,
blocks are further divided into sectors. Datasheets sometimes name blocks as
sectors and vice versa.
Common memory capacaties for SPI flashes are 512kB up to 8 MB of data, with
blocks of 64K. Sectors normally are 4K, if supported. The entire memory is
linear and can be written in random access, but erasing can only be done block-
or sectorwise; or by mass erase.
Common memory capacaties for SPI flashes are 512kB up to 8MB of data, where
blocks may be 64kB. Sectors can be e.g. 4kB, if supported. Many SPI flashes
have uniform block sizes, whereas others have non-uniform - the latter meaning
that e.g. the first 16 blocks are 4kB big, and the rest are 64kB.
The entire memory is linear and can be read and written in random access.
Erasing can only be done block- or sectorwise; or by mass erase.
SPI flashes can normally be erased from 100.000 up to 1.000.000 cycles before
they fail erasing.
they fail.
A clean SPI flash from factory have all bits in entire memory set to one. A
mass erase will reset the device to this state. Block or sector erasing will
put the area given by the sector or block to ones. Writing to a SPI flash
pulls ones to zeroes. Writing 0xFF to an address is simply a no-op. This way
of "nand-writing" is used considerably in spiffs.
put the all bits in the area given by the sector or block to ones. Writing to a
NOR flash pulls ones to zeroes. Writing 0xFF to an address is simply a no-op.
Writing 0b10101010 to a flash address holding 0b00001111 will yield 0b00001010.
This way of "write by nand" is used considerably in spiffs.
Common characteristics of NOR flashes are quick reads, but slow writes.
** Spiffs logical structure
Some terminology before proceeding. Physical blocks/sectors means sizes stated
in the datasheet. Logical blocks and pages is something the integrator choose.
** Blocks and pages
Spiffs is allocated to a part or all of the memory of the SPI flash device.
This area is divided into logical blocks, which in turn are divided into
logical pages. The boundary of a logical block must coincide with one or more
physical blocks. The sizes for logical blocks and logical pages always remain
the same, they are uniform.
Example: non-uniform flash mapped to spiffs with 128kB logical blocks
PHYSICAL FLASH BLOCKS SPIFFS LOGICAL BLOCKS: 128kB
+-----------------------+ - - - +-----------------------+
| Block 1 : 16kB | | Block 1 : 128kB |
+-----------------------+ | |
| Block 2 : 16kB | | |
+-----------------------+ | |
| Block 3 : 16kB | | |
+-----------------------+ | |
| Block 4 : 16kB | | |
+-----------------------+ | |
| Block 5 : 64kB | | |
+-----------------------+ - - - +-----------------------+
| Block 6 : 64kB | | Block 2 : 128kB |
+-----------------------+ | |
| Block 7 : 64kB | | |
+-----------------------+ - - - +-----------------------+
| Block 8 : 64kB | | Block 3 : 128kB |
+-----------------------+ | |
| Block 9 : 64kB | | |
+-----------------------+ - - - +-----------------------+
| ... | | ... |
A logical block is divided further into a number of logical pages. A page
defines the smallest data holding element known to spiffs. Hence, if a file
is created being one byte big, it will occupy one page for index and one page
for data - it will occupy 2 x size of a logical page on flash.
So it seems it is good to select a small page size.
Each page has a metadata header being normally 5 to 9 bytes. This said, a very
small page size will make metadata occupy a lot of the memory on the flash. A
page size of 64 bytes will waste 8-14% on metadata, while 256 bytes 2-4%.
So it seems it is good to select a big page size.
Also, spiffs uses a ram buffer being two times the page size. This ram buffer
is used for loading and manipulating pages, but it is also used for algorithms
to find free file ids, scanning the file system, etc. Having too small a page
size means less work buffer for spiffs, ending up in more reads operations and
eventually gives a slower file system.
Choosing the page size for the system involves many factors:
- How big is the logical block size
- What is the normal size of most files
- How much ram can be spent
- How much data (vs metadata) must be crammed into the file system
- How fast must spiffs be
- Other things impossible to find out
So, chosing the Optimal Page Size (tm) seems tricky, to say the least. Don't
fret - there is no optimal page size. This varies from how the target will use
spiffs. Use the golden rule:
~~~ Logical Page Size = Logical Block Size / 256 ~~~
This is a good starting point. The final page size can then be derived through
heuristical experimenting for us non-analytical minds.
** Objects, indices and look-ups
A file, or an object as called in spiffs, is identified by an object id.
Another YAFFS rip-off. This object id is a part of the page header. So, all
pages know to which object/file they belong - not counting the free pages.
An object is made up of two types of pages: object index pages and data pages.
Data pages contain the data written by user. Index pages contain metadata about
the object, more specifically what data pages are part of the object.
The page header also includes something called a span index. Let's say a file
is written covering three data pages. The first data page will then have span
index 0, the second span index 1, and the last data page will have span index
2. Simple as that.
Finally, each page header contain flags, telling if the page is used,
deleted, finalized, holds index or data, and more.
Object indices also have span indices, where an object index with span index 0
is referred to as the object index header. This page does not only contain
references to data pages, but also extra info such as object name, object size
in bytes, flags for file or directory, etc.
If one were to create a file covering three data pages, named e.g.
"spandex-joke.txt", given object id 12, it could look like this:
PAGE 0 <things to be unveiled soon>
PAGE 1 page header: [obj_id:12 span_ix:0 flags:USED|DATA]
<first data page of joke>
PAGE 2 page header: [obj_id:12 span_ix:1 flags:USED|DATA]
<second data page of joke>
PAGE 3 page header: [obj_id:545 span_ix:13 flags:USED|DATA]
<some data belonging to object 545, probably not very amusing>
PAGE 4 page header: [obj_id:12 span_ix:2 flags:USED|DATA]
<third data page of joke>
PAGE 5 page header: [obj_id:12 span_ix:0 flags:USED|INDEX]
obj ix header: [name:spandex-joke.txt size:600 bytes flags:FILE]
obj ix: [1 2 4]
Looking in detail at page 5, the object index header page, the object index
array refers to each data page in order, as mentioned before. The index of the
object index array correlates with the data page span index.
entry ix: 0 1 2
obj ix: [1 2 4]
| | |
PAGE 1, DATA, SPAN_IX 0 --------/ | |
PAGE 2, DATA, SPAN_IX 1 --------/ |
PAGE 4, DATA, SPAN_IX 2 --------/
Things to be unveiled in page 0 - well.. Spiffs is designed for systems low on
ram. We cannot keep a dynamic list on the whereabouts of each object index
header so we can find a file fast. There might not even be a heap! But, we do
not want to scan all page headers on the flash to find the object index header.
The first page(s) of each block contains the so called object look-up. These
are not normal pages, they do not have a header. Instead, they are arrays
pointing out what object-id the rest of all pages in the block belongs to.
By this look-up, only the first page(s) in each block must to scanned to find
the actual page which contains the object index header of the desired object.
The object lookup is redundant metadata. The assumption is that it presents
less overhead reading a full page of data to memory from each block and search
that, instead of reading a small amount of data from each page (i.e. the page
header) in all blocks. Each read operation from SPI flash normally contains
extra data as the read command itself and the flash address. Also, depending on
the underlying implementation, other criterions may need to be passed for each
read transaction, like mutexes and such.
The veiled example unveiled would look like this, with some extra pages:
PAGE 0 [ 12 12 545 12 12 34 34 4 0 0 0 0 ...]
PAGE 1 page header: [obj_id:12 span_ix:0 flags:USED|DATA] ...
PAGE 2 page header: [obj_id:12 span_ix:1 flags:USED|DATA] ...
PAGE 3 page header: [obj_id:545 span_ix:13 flags:USED|DATA] ...
PAGE 4 page header: [obj_id:12 span_ix:2 flags:USED|DATA] ...
PAGE 5 page header: [obj_id:12 span_ix:0 flags:USED|INDEX] ...
PAGE 6 page header: [obj_id:34 span_ix:0 flags:USED|DATA] ...
PAGE 7 page header: [obj_id:34 span_ix:1 flags:USED|DATA] ...
PAGE 8 page header: [obj_id:4 span_ix:1 flags:USED|INDEX] ...
PAGE 9 page header: [obj_id:23 span_ix:0 flags:DELETED|INDEX] ...
PAGE 10 page header: [obj_id:23 span_ix:0 flags:DELETED|DATA] ...
PAGE 11 page header: [obj_id:23 span_ix:1 flags:DELETED|DATA] ...
PAGE 12 page header: [obj_id:23 span_ix:2 flags:DELETED|DATA] ...
...
Ok, so why are page 9 to 12 marked as 0 when they belong to object id 23? These
pages are deleted, so this is marked both in page header flags and in the look
up. This is an example where spiffs uses NOR flashes "nand-way" of writing.
As a matter of fact, there are two object id's which are special:
obj id 0 (all bits zeroes) - indicates a deleted page in object look up
obj id 0xff.. (all bits ones) - indicates a free page in object look up
Actually, the object id's have another quirk: if the most significant bit is
set, this indicates an object index page. If the most significant bit is zero,
this indicates a data page. So to be fully correct, page 0 in above example
would look like this:
PAGE 0 [ 12 12 545 12 *12 34 34 *4 0 0 0 0 ...]
where the asterisk means the msb is set.
** TODO
This is another way to speed up the searches when looking for object indices.
By looking on the object id's msb in the object lookup, it is also possible
whether the page is an object index page or a data page.

69
src/default/spiffs_config.h
View File

@ -8,13 +8,15 @@
#ifndef SPIFFS_CONFIG_H_
#define SPIFFS_CONFIG_H_
// following includes are for the linux test build of spiffs
// this may/should/must be removed/altered/replaced in your target
// ----------- 8< ------------
// Following includes are for the linux test build of spiffs
// These may/should/must be removed/altered/replaced in your target
#include "params_test.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stddef.h>
// ----------- >8 ------------
// compile time switches
@ -48,6 +50,8 @@
// Garbage collecting examines all pages in a block which and sums up
// to a block score. Deleted pages normally gives positive score and
// used pages normally gives a negative score (as these must be moved).
// To have a fair wear-leveling, the erase age is also included in score,
// whose factor normally is the most positive.
// The larger the score, the more likely it is that the block will
// picked for garbage collection.
@ -65,48 +69,63 @@
// size of buffer on stack used when copying data
#define SPIFFS_COPY_BUFFER_STACK (64)
// SPIFFS_LOCK and SPIFFS_UNLOCK protects spiffs from reentrancy on api level
// These must be defined on a multithreaded system
// types and constants
// define this to entering a mutex if you're running on a multithreaded system
#define SPIFFS_LOCK(fs)
// define this to exiting a mutex if you're running on a multithreaded system
#define SPIFFS_UNLOCK(fs)
// spiffs file descriptor index type
typedef s16_t spiffs_file;
// spiffs file descriptor attributes
typedef u32_t spiffs_attr;
// spiffs file descriptor mode
typedef u32_t spiffs_mode;
// block index type
typedef u16_t spiffs_block_ix; // (address-phys_addr) / block_size
// page index type
typedef u16_t spiffs_page_ix; // (address-phys_addr) / page_size
// object id type - most significant bit is reserved for index flag
typedef u16_t spiffs_obj_id;
// object span index type
typedef u16_t spiffs_span_ix;
// object type
typedef u8_t spiffs_obj_type;
// Enable if only one spiffs instance with constant configuration will exist
// on the target, this will reduce calculations, flash and memory accesses.
#define SPIFFS_SINGLETON 0
// enable if only one spiffs instance with constant configuration will exist
// on the target, this will reduce calculations, flash and memory accesses
//#define SPIFFS_SINGLETON
#ifdef SPIFFS_SINGLETON
#if SPIFFS_SINGLETON
// instead of giving parameters in config struct, singleton build must
// give parameters in defines below
#define SPIFFS_CFG_PHYS_SZ(ignore) (1024*1024*1)
#define SPIFFS_CFG_PHYS_SZ(ignore) (1024*1024*2)
#define SPIFFS_CFG_PHYS_ERASE_SZ(ignore) (65536)
#define SPIFFS_CFG_PHYS_ADDR(ignore) (0)
#define SPIFFS_CFG_LOG_PAGE_SZ(ignore) (256)
#define SPIFFS_CFG_LOG_BLOCK_SZ(ignore) (65536)
#endif
// Set SPFIFS_TEST_VISUALISATION to non-zero to enable SPIFFS_vis function
// in the api. This function will visualize all filesystem using given printf
// function.
#define SPIFFS_TEST_VISUALISATION 1
#if SPIFFS_TEST_VISUALISATION
#define spiffs_printf(...) printf(__VA_ARGS__)
// spiffs_printf argument for a free page
#define SPIFFS_TEST_VIS_FREE_STR "_"
// spiffs_printf argument for a deleted page
#define SPIFFS_TEST_VIS_DELE_STR "/"
// spiffs_printf argument for an index page for given object id
#define SPIFFS_TEST_VIS_INDX_STR(id) "i"
// spiffs_printf argument for a data page for given object id
#define SPIFFS_TEST_VIS_DATA_STR(id) "d"
#endif
// types and constants
// spiffs file descriptor index type
typedef s16_t spiffs_file;
// spiffs file descriptor attributes
typedef u32_t spiffs_attr;
// spiffs file descriptor mode
typedef u32_t spiffs_mode;
// block index type
typedef u16_t spiffs_block_ix; // (address-phys_addr) / block_size
// page index type
typedef u16_t spiffs_page_ix; // (address-phys_addr) / page_size
// object id type - most significant bit is reserved for index flag
typedef u16_t spiffs_obj_id;
// object span index type
typedef u16_t spiffs_span_ix;
// object type
typedef u8_t spiffs_obj_type;
#endif /* SPIFFS_CONFIG_H_ */

25
src/spiffs.h
View File

@ -34,6 +34,8 @@
#define SPIFFS_ERR_INDEX_FREE -10018
#define SPIFFS_ERR_INDEX_LU -10019
#define SPIFFS_ERR_INDEX_INVALID -10020
#define SPIFFS_ERR_NOT_WRITABLE -10021
#define SPIFFS_ERR_NOT_READABLE -10022
#define SPIFFS_ERR_INTERNAL -10050
@ -122,7 +124,7 @@ typedef struct {
spiffs_write hal_write_f;
// physical erase function
spiffs_erase hal_erase_f;
#ifndef SPIFFS_SINGLETON
#if SPIFFS_SINGLETON == 0
// physical size of the spi flash
u32_t phys_size;
// physical offset in spi flash used for spiffs,
@ -345,8 +347,29 @@ void SPIFFS_close(spiffs *fs, spiffs_file fh);
*/
s32_t SPIFFS_errno(spiffs *fs);
/**
* Opens a directory stream corresponding to the given name.
* The stream is positioned at the first entry in the directory.
* On hydrogen builds the name argument is ignored as hydrogen builds always correspond
* to a flat file structure - no directories.
* @param fs the file system struct
* @param name the name of the directory
* @param d pointer the directory stream to be populated
*/
spiffs_DIR *SPIFFS_opendir(spiffs *fs, const char *name, spiffs_DIR *d);
/**
* Closes a directory stream
* @param d the directory stream to close
*/
s32_t SPIFFS_closedir(spiffs_DIR *d);
/**
* Reads a directory into given spifs_dirent struct.
* @param d pointer to the directory stream
* @param e the dirent struct to be populated
* @returns null if error or end of stream, else given dirent is returned
*/
struct spiffs_dirent *SPIFFS_readdir(spiffs_DIR *d, struct spiffs_dirent *e);
/**

15
src/spiffs_hydrogen.c
View File

@ -175,6 +175,11 @@ s32_t SPIFFS_read(spiffs *fs, spiffs_file fh, void *buf, s32_t len) {
res = spiffs_fd_get(fs, fh, &fd);
SPIFFS_API_CHECK_RES_UNLOCK(fs, res);
if ((fd->mode & SPIFFS_RDONLY) == 0) {
res = SPIFFS_ERR_NOT_READABLE;
SPIFFS_API_CHECK_RES_UNLOCK(fs, res);
}
#if SPIFFS_CACHE_WR
spiffs_fflush_cache(fs, fh);
#endif
@ -235,6 +240,11 @@ s32_t SPIFFS_write(spiffs *fs, spiffs_file fh, void *buf, s32_t len) {
res = spiffs_fd_get(fs, fh, &fd);
SPIFFS_API_CHECK_RES_UNLOCK(fs, res);
if ((fd->mode & SPIFFS_WRONLY) == 0) {
res = SPIFFS_ERR_NOT_WRITABLE;
SPIFFS_API_CHECK_RES_UNLOCK(fs, res);
}
offset = fd->fdoffset;
#if SPIFFS_CACHE_WR
@ -423,6 +433,11 @@ s32_t SPIFFS_fremove(spiffs *fs, spiffs_file fh) {
res = spiffs_fd_get(fs, fh, &fd);
SPIFFS_API_CHECK_RES_UNLOCK(fs, res);
if ((fd->mode & SPIFFS_WRONLY) == 0) {
res = SPIFFS_ERR_NOT_WRITABLE;
SPIFFS_API_CHECK_RES_UNLOCK(fs, res);
}
#if SPIFFS_CACHE_WR
spiffs_cache_fd_release(fs, fd->cache_page);
#endif

10
src/spiffs_nucleus.h
View File

@ -131,7 +131,7 @@
#define SPIFFS_OBJ_ID_DELETED ((spiffs_obj_id)0)
#define SPIFFS_OBJ_ID_FREE ((spiffs_obj_id)-1)
#ifndef SPIFFS_SINGLETON
#if SPIFFS_SINGLETON == 0
#define SPIFFS_CFG_LOG_PAGE_SZ(fs) \
((fs)->cfg.log_page_size)
#define SPIFFS_CFG_LOG_BLOCK_SZ(fs) \
@ -426,6 +426,14 @@ typedef s32_t (*spiffs_visitor_f)(spiffs *fs, spiffs_obj_id id, spiffs_block_ix
#define _spiffs_wr(fs, op, fh, addr, len, src) \
spiffs_phys_wr((fs), (addr), (len), (src))
#endif
#ifndef MIN
#define MIN(a,b) ((a) < (b) ? (a) : (b))
#endif
#ifndef MAX
#define MAX(a,b) ((a) > (b) ? (a) : (b))
#endif
// ---------------
s32_t spiffs_phys_rd(

3
src/test/params_test.h
View File

@ -18,9 +18,6 @@
#define ASSERT(c, m) real_assert((c),(m), __FILE__, __LINE__);
#define MIN(a,b) ((a) < (b) ? (a) : (b))
#define MAX(a,b) ((a) > (b) ? (a) : (b))
void real_assert(int c, const char *n, const char *file, int l);
typedef signed int s32_t;

Write Preview
Loading…
Cancel
Save