stm32f4x/codec2/README_fdmdv.txt

README_fdmdv.txt
David Rowe
Created March 2012

Introduction
------------

A 1400 bit/s (nominal) Frequency Division Multiplexed Digital Voice
(FDMDV) modem based on [1].  Used for digital audio over HF SSB.

The FDMDV modem was first implemented in GNU Octave, then ported to C.
Algorithm development is generally easier in Octave, but for real time
work we need the C version.  Automated units tests ensure the
operation of the Octave and C versions are identical.

Quickstart
----------

Built as part of codec2-dev, see README for build instructions.

1. Generate some test bits and modulate them:

    $ ./fdmdv_get_test_bits test.c2 1400
    $ ./fdmdv_mod test.c2 test.raw
    $ play -r 8000 -s -2 test.raw

2. Two seconds of test frame data modulated and sent out of sound device:

    $ ./fdmdv_get_test_bits - 2800 | ./fdmdv_mod - - | play -t raw -r 8000 -s -2 -

3. Send 14000 modulated bits (10 seconds) to the demod and count errors:

    $  ./fdmdv_get_test_bits - 14000 | ./fdmdv_mod - - | ./fdmdv_demod - - 14 demod_dump.txt | ./fdmdv_put_test_bits -

    Use Octave to look at plots of 1 second (1400 bits) of modem operation:

    $ cd ../octave
    $ octave
    octave:1> fdmdv_demod_c("../src/demod_dump.txt",14000)

4. Run Octave simulation of entire modem and AWGN channel:

    $ cd ../octave
    $ octave
    octave:1> fdmdv_ut

5. NOTE: If you would like to play modem samples over the air please
   convert the 8 kHz samples to 48 kHz.  Many PC sound cards have
   wildly inaccurate sample clock rates when set to 8 kHz, but seem to
   perform OK when set for 48 kHz.  If playing and recording files you
   can use the sox utility:

   $ sox -r 8000 -s -2 modem_sample_8kHz.raw -r 48000 modem_sample_48kHz.wav

   For real time applications, the fdmdv.[ch] library includes functions to
   convert between 48 and 8 kHz sample rates.

6. Send 20 seconds at 2000 bit/s (20 carriers) to demod and count errors:

   $ ./fdmdv_get_test_bits - 20000 20 | ./fdmdv_mod - - 20 | ./fdmdv_demod - - 20 | ./fdmdv_put_test_bits - 20

7. Test with timing slips due to sample clock offset of 1000ppm:

   $ ./fdmdv_get_test_bits - 30000 | ./fdmdv_mod - - | sox -t raw -r 8000 -s -2 - -t raw -r 7990 - | ./fdmdv_demod - - 14 demod_dump.txt | ./fdmdv_put_test_bits -

   octave:98> fdmdv_demod_c("../build_linux/src/demod_dump.txt",28000)
   27552 bits  0 errors  BER: 0.0000

References
----------

[1] http://n1su.com/fdmdv/FDMDV_Docs_Rel_1_4b.pdf
[2] http://n1su.com/fdmdv/
[3] http://www.rowetel.com/blog/?p=2433 "Testing a FDMDV Modem"
[4] http://www.rowetel.com/blog/?p=2458 "FDMDV Modem Page" on David's web site

C Code
------

src/fdmdv_mod.c - C version of modulator that takes a file of bits and
                  converts it to a raw file of modulated samples.

src/fdmdv_demod.c - C version of demodulator that takes a raw file of
                    modulated samples and outputs a file of bits.
                    Optionally dumps demod states to a text file which
                    can be plotted using the Octave script
                    fdmdv_demod_c.m

src/codec2_fdmdv.h - Header file that exposes FDMDV C API functions.  Include
                    this file in your application program.

src/fdmdv.c - C functions that implement the FDMDV modem.

src/fdmdv-internal.h - internal states and constants for FDMDV modem,
                       shouldn't be exposed to application program.


unittest/tfdmdv.c - Used to conjunction with unittest/tfdmdv.m to
                    automatically test C FDMDV functions against
                    Octave versions.

Octave Scripts
--------------

(Note these require some Octave packages to be installed, see
octave/README.txt).

fdmdv.m - Functions and variables that implement the Octave version of
          the FDMDV modem.

fdmdv_ut.m - Unit test for fdmdv Octave code, useful while
             developing algorithm.  Includes tx/rx plus basic channel
             simulation.

             Typical run:

               octave:6> fdmdv_ut
               Eb/No (meas): 7.30 (8.29) dB
               bits........: 2464
               errors......: 20
               BER.........: 0.0081
               PAPR........: 13.54 dB
               SNR.........: 4.0 dB

               It also outputs lots of nice plots that show the
	       operation of the modem.

               For a 1400 bit/s DQPSK modem we expect about 1% BER for
               Eb/No = 7.3dB, which corresponds to SNR = 4dB (3kHz
               noise BW). The extra dB of measured power is due to the
               DBPSK pilot. Currently the noise generation code
               doesn't take the pilot power into account, so in this
               example the real SNR is actually 5dB.

fdmdv_mod.m - Octave version of modulator that outputs a raw file.
              The modulator is driven by a test frame of bits.  This
              can then be played over a real channel or through a
              channel simulator like PathSim.  The sample rate can be
              changed using "sox" to simulate differences in tx/rx
              sample clocks.

	      To generate 10 seconds of modulated signal:

                octave:8> fdmdv_mod("test.raw",1400*10);

fdmdv_demod.m - Demodulator program that takes a raw file as input,
                and works out the bit error rate using the known test
                frame.  Can be used to test the demod performs with
                off-air signals, or signals that have been passed
                through a channel simulator.

		To demodulate 2 seconds of the test.raw file generated
		above:

                octave:9> fdmdv_demod("test.raw",1400*2);
                2464 bits  0 errors  BER: 0.0000

                It also produces several plots showing the internal
		states of the demod.  Useful for debugging and
		observing what happens with various channels.

fdmdv_demod_c.m - Takes an output text file from the C demod
                  fdmdv_demod.c and produces plots and measures BER.
                  Useful for evaluating fdmdv_demod.c performance.
                  The plots produced are identical to the Octave
                  version fdmdv_demod.m, allowing direct comparison of
                  the C and Octave versions.

tfdmdv.m - Automatic tests that compare the Octave and C versions of
           the FDMDV modem functions.  First run unittest/tfdmdv, this
           will generate a text file with test vectors from the C
           version.  Then run the Octave script tfdmdv and it will
           generate Octave versions of the test vectors and compare
           each vector with the C equivalent.  Its plots the vectors
           and and errors (green).  Its also produces an automatic
           check list based on test results.  If the Octave or C modem
           code is changed, this script should be used to ensure the
           C and Octave versions remain identical.

Modelling sample clock errors using sox
---------------------------------------

This introduces a simulated 1000ppm error:

  sox -r 8000 -s -2 mod_dqpsk.raw -s -2 mod_dqpsk_8008hz.raw rate -h 8008

TODO
----

[ ] implement ppm measurements in fdmdv_get_demod_stats()
[ ] try interfering sine wave
    + maybe swept
    + does modem fall over?
[ ] try non-flat channel, e.g. 3dB difference between hi and low tones
    + make sure all estimators keep working
[ ] test rx level sensitivity, i.e. 0 to 20dB attenuation
[ ] make fine freq indep of amplitude
    + use angle rather than imag coord
[ ] document use of fdmdv_ut and fdmdv_demod + PathSim
[ ] more positive form of sync reqd for DV frames?
    + like using coarse_fine==1 to decode valid DV frame bit?
    + when should we start decoding?
[ ] more robust track/acquite state machine?
    + e.g. hang on thru the fades?
[ ] PAPR idea
    + automatically tweak phases to reduce PAPR, e.g. slow variations in freq...
[ ] why is pilot noise_est twice as big as other carriers