Note: this is not an announcement of FFglitch version 101 being released.

Welcome to the first post in a series that tries to explain what FFglitch is, what it can do, and how to use it. Hopefully this series will contain more than just one post.

Introduction

As you can read in the What? page, FFglitch works in three steps:

  1. Export data from media file
  2. Glitch data however you see fit
  3. Apply glitched data and generate a new file

You would normally use any media file you want to glitch as input, but sometimes there is also step 0, which is to generate an input file to work with. In this post, I’ll start with step 0.

0 - Generate input file

One file I like to work with is an empty JPEG file. It’s basically just a small JPEG file, 16x16 pixels, that can be used as a white canvas to work on.

If you’re on Linux or macOS, you can generate it with this command line:

$ ffmpeg -s 16x16 -pix_fmt monow -f rawvideo -i /dev/zero -flags +bitexact \
         -huffman default -pix_fmt yuvj420p -vframes 1 -y empty.jpeg

The result is this amazing 16x16 pixels white JPEG file:

empty JPEG 16x16 pixels white JPEG (not to scale)

You can right-click and download the image above if you don’t feel like creating it yourself.

After creating the file, I like to take a hex dump of its contents, just to see what it looks like:

$ hexdump -C empty.jpeg
00000000  ff d8 ff db 00 43 00 08  04 04 04 04 04 05 05 05  |.....C..........|
00000010  05 05 05 06 06 06 06 06  06 06 06 06 06 06 06 06  |................|
00000020  07 07 07 08 08 08 07 07  07 06 06 07 07 08 08 08  |................|
00000030  08 09 09 09 08 08 08 08  09 09 0a 0a 0a 0c 0c 0b  |................|
00000040  0b 0e 0e 0e 11 11 14 ff  c4 01 a2 00 00 01 05 01  |................|
00000050  01 01 01 01 01 00 00 00  00 00 00 00 00 01 02 03  |................|
00000060  04 05 06 07 08 09 0a 0b  01 00 03 01 01 01 01 01  |................|
00000070  01 01 01 01 00 00 00 00  00 00 01 02 03 04 05 06  |................|
00000080  07 08 09 0a 0b 10 00 02  01 03 03 02 04 03 05 05  |................|
00000090  04 04 00 00 01 7d 01 02  03 00 04 11 05 12 21 31  |.....}........!1|
000000a0  41 06 13 51 61 07 22 71  14 32 81 91 a1 08 23 42  |A..Qa."q.2....#B|
000000b0  b1 c1 15 52 d1 f0 24 33  62 72 82 09 0a 16 17 18  |...R..$3br......|
000000c0  19 1a 25 26 27 28 29 2a  34 35 36 37 38 39 3a 43  |..%&'()*456789:C|
000000d0  44 45 46 47 48 49 4a 53  54 55 56 57 58 59 5a 63  |DEFGHIJSTUVWXYZc|
000000e0  64 65 66 67 68 69 6a 73  74 75 76 77 78 79 7a 83  |defghijstuvwxyz.|
000000f0  84 85 86 87 88 89 8a 92  93 94 95 96 97 98 99 9a  |................|
00000100  a2 a3 a4 a5 a6 a7 a8 a9  aa b2 b3 b4 b5 b6 b7 b8  |................|
00000110  b9 ba c2 c3 c4 c5 c6 c7  c8 c9 ca d2 d3 d4 d5 d6  |................|
00000120  d7 d8 d9 da e1 e2 e3 e4  e5 e6 e7 e8 e9 ea f1 f2  |................|
00000130  f3 f4 f5 f6 f7 f8 f9 fa  11 00 02 01 02 04 04 03  |................|
00000140  04 07 05 04 04 00 01 02  77 00 01 02 03 11 04 05  |........w.......|
00000150  21 31 06 12 41 51 07 61  71 13 22 32 81 08 14 42  |!1..AQ.aq."2...B|
00000160  91 a1 b1 c1 09 23 33 52  f0 15 62 72 d1 0a 16 24  |.....#3R..br...$|
00000170  34 e1 25 f1 17 18 19 1a  26 27 28 29 2a 35 36 37  |4.%.....&'()*567|
00000180  38 39 3a 43 44 45 46 47  48 49 4a 53 54 55 56 57  |89:CDEFGHIJSTUVW|
00000190  58 59 5a 63 64 65 66 67  68 69 6a 73 74 75 76 77  |XYZcdefghijstuvw|
000001a0  78 79 7a 82 83 84 85 86  87 88 89 8a 92 93 94 95  |xyz.............|
000001b0  96 97 98 99 9a a2 a3 a4  a5 a6 a7 a8 a9 aa b2 b3  |................|
000001c0  b4 b5 b6 b7 b8 b9 ba c2  c3 c4 c5 c6 c7 c8 c9 ca  |................|
000001d0  d2 d3 d4 d5 d6 d7 d8 d9  da e2 e3 e4 e5 e6 e7 e8  |................|
000001e0  e9 ea f2 f3 f4 f5 f6 f7  f8 f9 fa ff c0 00 11 08  |................|
000001f0  00 10 00 10 03 01 22 00  02 11 00 03 11 00 ff da  |......".........|
00000200  00 0c 03 01 00 02 11 03  11 00 3f 00 f7 fa 28 a2  |..........?...(.|
00000210  80 3f ff d9                                       |.?..|
00000214

You could open it in a hex editor and start hacking away, but let’s see what FFglitch has to say about it.

1 - Export data from media file

With FFglitch, you have to choose which feature of the codec you want to glitch. This is codec-specific, and you can see more about it in the What? page.

In this example, we’ll work on the quantized DCT coefficients. More specifically, only the DC coefficients (feature name: q_dc).

We’ll export this feature from empty.jpeg into a JSON file.

$ ffedit empty.jpeg -f q_dc -e empty.json

Now open empty.json in your favorite text editor, and this is what you’ll see:

{
  "ffedit_version":"ffglitch-0.7.001",
  "filename":"empty.jpeg",
  "sha1sum":"620ffe3810380fcd07e19a722fef8cf96a751eb3",
  "features":[
    "q_dc"
  ],
  "streams":[
    {
      "frames":[
        {
          "pkt_pos":0,
          "pts":0,
          "dts":0,
          "q_dc":{
            "data":[
              [
                [ 127, 0 ],
                [ 0, 0 ]
              ],
              [
                [ 0 ]
              ],
              [
                [ 0 ]
              ]
            ],
            "luma_max":2047,
            "chroma_max":2047
          }
        }
      ]
    }
  ]
}

There’s a lot of stuff there, but you just need to focus on the “data” part:

            "data":[
              [
                [ 127, 0 ],
                [ 0, 0 ]
              ],
              [
                [ 0 ]
              ],
              [
                [ 0 ]
              ]
            ],

So we have one pair of [ ] that encompasses all the data, which consists of three pairs of [ ]. The first one contains two lines, with two numbers inside each one of them. The second and the third one contain just one line each, with just one number inside each of them.

To understand what this means, you have to understand what a macroblock is and the pixel format used by this JPEG file (yuv420p). But I will not explain this in this post :). Instead I’ll just comment on the data.

            "data":[
              [             ; "Y plane (contains four 8x8 blocks)"
                [ 127, 0 ],
                [ 0, 0 ]
              ],
              [             ; "U plane (contains one 8x8 block)"
                [ 0 ]
              ],
              [             ; "V plane (contains one 8x8 block)"
                [ 0 ]
              ]
            ],

2 - Glitch data however you see fit

We’ll start out by just experimenting on changing the values in the data structure and seeing what gives in the output. For example, let’s change that 127 to a -128. We’ll end up with:

{
  "ffedit_version":"ffglitch-0.7.001",
  "filename":"empty.jpeg",
  "sha1sum":"620ffe3810380fcd07e19a722fef8cf96a751eb3",
  "features":[
    "q_dc"
  ],
  "streams":[
    {
      "frames":[
        {
          "pkt_pos":0,
          "pts":0,
          "dts":0,
          "q_dc":{
            "data":[
              [
                [ -128, 0 ],
                [ 0, 0 ]
              ],
              [
                [ 0 ]
              ],
              [
                [ 0 ]
              ]
            ],
            "luma_max":2047,
            "chroma_max":2047
          }
        }
      ]
    }
  ]
}

Now save the file with another filename (glitch.json), and let’s see what this changed in the JPEG file.

3 - Apply glitched data and generate a new file

Now we have to tell FFglitch to apply the changes we did in the JSON file back to the JPEG file. Actually, we won’t overwrite the original file (empty.jpeg). Instead, we’ll create a new file, but ask FFglitch to get the modified values for the DC coefficients from the glitched JSON file (glitch.json). We’ll call this new file output1.jpeg:

$ ffedit empty.jpeg -f q_dc -a empty.json output1.jpeg

Now watch in awe what FFglitch did to our 16x16 pixels white JPEG canvas:

glitched JPEG 16x16 pixels no-longer-white JPEG (not to scale)

If you take a hex dump of the new file, you’ll get this:

$ hexdump -C output1.jpeg 
00000000  ff d8 ff db 00 43 00 08  04 04 04 04 04 05 05 05  |.....C..........|
00000010  05 05 05 06 06 06 06 06  06 06 06 06 06 06 06 06  |................|
00000020  07 07 07 08 08 08 07 07  07 06 06 07 07 08 08 08  |................|
00000030  08 09 09 09 08 08 08 08  09 09 0a 0a 0a 0c 0c 0b  |................|
00000040  0b 0e 0e 0e 11 11 14 ff  c4 01 a2 00 00 01 05 01  |................|
00000050  01 01 01 01 01 00 00 00  00 00 00 00 00 01 02 03  |................|
00000060  04 05 06 07 08 09 0a 0b  01 00 03 01 01 01 01 01  |................|
00000070  01 01 01 01 00 00 00 00  00 00 01 02 03 04 05 06  |................|
00000080  07 08 09 0a 0b 10 00 02  01 03 03 02 04 03 05 05  |................|
00000090  04 04 00 00 01 7d 01 02  03 00 04 11 05 12 21 31  |.....}........!1|
000000a0  41 06 13 51 61 07 22 71  14 32 81 91 a1 08 23 42  |A..Qa."q.2....#B|
000000b0  b1 c1 15 52 d1 f0 24 33  62 72 82 09 0a 16 17 18  |...R..$3br......|
000000c0  19 1a 25 26 27 28 29 2a  34 35 36 37 38 39 3a 43  |..%&'()*456789:C|
000000d0  44 45 46 47 48 49 4a 53  54 55 56 57 58 59 5a 63  |DEFGHIJSTUVWXYZc|
000000e0  64 65 66 67 68 69 6a 73  74 75 76 77 78 79 7a 83  |defghijstuvwxyz.|
000000f0  84 85 86 87 88 89 8a 92  93 94 95 96 97 98 99 9a  |................|
00000100  a2 a3 a4 a5 a6 a7 a8 a9  aa b2 b3 b4 b5 b6 b7 b8  |................|
00000110  b9 ba c2 c3 c4 c5 c6 c7  c8 c9 ca d2 d3 d4 d5 d6  |................|
00000120  d7 d8 d9 da e1 e2 e3 e4  e5 e6 e7 e8 e9 ea f1 f2  |................|
00000130  f3 f4 f5 f6 f7 f8 f9 fa  11 00 02 01 02 04 04 03  |................|
00000140  04 07 05 04 04 00 01 02  77 00 01 02 03 11 04 05  |........w.......|
00000150  21 31 06 12 41 51 07 61  71 13 22 32 81 08 14 42  |!1..AQ.aq."2...B|
00000160  91 a1 b1 c1 09 23 33 52  f0 15 62 72 d1 0a 16 24  |.....#3R..br...$|
00000170  34 e1 25 f1 17 18 19 1a  26 27 28 29 2a 35 36 37  |4.%.....&'()*567|
00000180  38 39 3a 43 44 45 46 47  48 49 4a 53 54 55 56 57  |89:CDEFGHIJSTUVW|
00000190  58 59 5a 63 64 65 66 67  68 69 6a 73 74 75 76 77  |XYZcdefghijstuvw|
000001a0  78 79 7a 82 83 84 85 86  87 88 89 8a 92 93 94 95  |xyz.............|
000001b0  96 97 98 99 9a a2 a3 a4  a5 a6 a7 a8 a9 aa b2 b3  |................|
000001c0  b4 b5 b6 b7 b8 b9 ba c2  c3 c4 c5 c6 c7 c8 c9 ca  |................|
000001d0  d2 d3 d4 d5 d6 d7 d8 d9  da e2 e3 e4 e5 e6 e7 e8  |................|
000001e0  e9 ea f2 f3 f4 f5 f6 f7  f8 f9 fa ff c0 00 11 08  |................|
000001f0  00 10 00 10 03 01 22 00  02 11 00 03 11 00 ff da  |......".........|
00000200  00 0c 03 01 00 02 11 03  11 00 3f 00 f9 fe 8a 28  |..........?....(|
00000210  a0 0f ff d9                                       |....|
00000214

And if you look at the differences between the original file and the glitched file, you’ll get this:

 000001d0  d2 d3 d4 d5 d6 d7 d8 d9  da e2 e3 e4 e5 e6 e7 e8  |................|
 000001e0  e9 ea f2 f3 f4 f5 f6 f7  f8 f9 fa ff c0 00 11 08  |................|
 000001f0  00 10 00 10 03 01 22 00  02 11 00 03 11 00 ff da  |......".........|
-00000200  00 0c 03 01 00 02 11 03  11 00 3f 00 f7 fa 28 a2  |..........?...(.|
-00000210  80 3f ff d9                                       |.?..|
+00000200  00 0c 03 01 00 02 11 03  11 00 3f 00 f9 fe 8a 28  |..........?....(|
+00000210  a0 0f ff d9                                       |....|
 00000214

Notice the values that have changed at the end of the file:

- [...] f7 fa 28 a2 80 3f [...]
+ [...] f9 fe 8a 28 a0 0f [...]

This means “change white to black”, but it would have been hard to get the bitstream change right if it had been done with a hex editor…

Conclusion

We could have just used the Bucket tool in Microsoft Paint to achieve the same effect. But it’s more fun with FFglitch.