FFglitch 101
Note: this is not an announcement of FFglitch version 101 being released.
Welcome to the first post in a series that tries to explain what FFglitch is, what it can do, and how to use it. Hopefully this series will contain more than just one post.
Introduction
As you can read in the What? page, FFglitch works in three steps:
- Export data from media file
- Glitch data however you see fit
- Apply glitched data and generate a new file
You would normally use any media file you want to glitch as input, but sometimes there is also step 0, which is to generate an input file to work with. In this post, I’ll start with step 0.
0 - Generate input file
One file I like to work with is an empty JPEG file. It’s basically just a small JPEG file, 16x16 pixels, that can be used as a white canvas to work on.
If you’re on Linux or macOS, you can generate it with this command line:
$ ffgac -s 16x16 -pix_fmt monow -f rawvideo -i /dev/zero -flags +bitexact \
-huffman default -pix_fmt yuvj420p -vframes 1 -y empty.jpeg
The result is this amazing 16x16 pixels white JPEG file:
16x16 pixels white JPEG (not to scale)
You can right-click and download the image above if you don’t feel like creating it yourself.
After creating the file, I like to take a hex dump of its contents, just to see what it looks like:
$ hexdump -C empty.jpeg
00000000 ff d8 ff db 00 43 00 08 04 04 04 04 04 05 05 05 |.....C..........|
00000010 05 05 05 06 06 06 06 06 06 06 06 06 06 06 06 06 |................|
00000020 07 07 07 08 08 08 07 07 07 06 06 07 07 08 08 08 |................|
00000030 08 09 09 09 08 08 08 08 09 09 0a 0a 0a 0c 0c 0b |................|
00000040 0b 0e 0e 0e 11 11 14 ff c4 01 a2 00 00 01 05 01 |................|
00000050 01 01 01 01 01 00 00 00 00 00 00 00 00 01 02 03 |................|
00000060 04 05 06 07 08 09 0a 0b 01 00 03 01 01 01 01 01 |................|
00000070 01 01 01 01 00 00 00 00 00 00 01 02 03 04 05 06 |................|
00000080 07 08 09 0a 0b 10 00 02 01 03 03 02 04 03 05 05 |................|
00000090 04 04 00 00 01 7d 01 02 03 00 04 11 05 12 21 31 |.....}........!1|
000000a0 41 06 13 51 61 07 22 71 14 32 81 91 a1 08 23 42 |A..Qa."q.2....#B|
000000b0 b1 c1 15 52 d1 f0 24 33 62 72 82 09 0a 16 17 18 |...R..$3br......|
000000c0 19 1a 25 26 27 28 29 2a 34 35 36 37 38 39 3a 43 |..%&'()*456789:C|
000000d0 44 45 46 47 48 49 4a 53 54 55 56 57 58 59 5a 63 |DEFGHIJSTUVWXYZc|
000000e0 64 65 66 67 68 69 6a 73 74 75 76 77 78 79 7a 83 |defghijstuvwxyz.|
000000f0 84 85 86 87 88 89 8a 92 93 94 95 96 97 98 99 9a |................|
00000100 a2 a3 a4 a5 a6 a7 a8 a9 aa b2 b3 b4 b5 b6 b7 b8 |................|
00000110 b9 ba c2 c3 c4 c5 c6 c7 c8 c9 ca d2 d3 d4 d5 d6 |................|
00000120 d7 d8 d9 da e1 e2 e3 e4 e5 e6 e7 e8 e9 ea f1 f2 |................|
00000130 f3 f4 f5 f6 f7 f8 f9 fa 11 00 02 01 02 04 04 03 |................|
00000140 04 07 05 04 04 00 01 02 77 00 01 02 03 11 04 05 |........w.......|
00000150 21 31 06 12 41 51 07 61 71 13 22 32 81 08 14 42 |!1..AQ.aq."2...B|
00000160 91 a1 b1 c1 09 23 33 52 f0 15 62 72 d1 0a 16 24 |.....#3R..br...$|
00000170 34 e1 25 f1 17 18 19 1a 26 27 28 29 2a 35 36 37 |4.%.....&'()*567|
00000180 38 39 3a 43 44 45 46 47 48 49 4a 53 54 55 56 57 |89:CDEFGHIJSTUVW|
00000190 58 59 5a 63 64 65 66 67 68 69 6a 73 74 75 76 77 |XYZcdefghijstuvw|
000001a0 78 79 7a 82 83 84 85 86 87 88 89 8a 92 93 94 95 |xyz.............|
000001b0 96 97 98 99 9a a2 a3 a4 a5 a6 a7 a8 a9 aa b2 b3 |................|
000001c0 b4 b5 b6 b7 b8 b9 ba c2 c3 c4 c5 c6 c7 c8 c9 ca |................|
000001d0 d2 d3 d4 d5 d6 d7 d8 d9 da e2 e3 e4 e5 e6 e7 e8 |................|
000001e0 e9 ea f2 f3 f4 f5 f6 f7 f8 f9 fa ff c0 00 11 08 |................|
000001f0 00 10 00 10 03 01 22 00 02 11 00 03 11 00 ff da |......".........|
00000200 00 0c 03 01 00 02 11 03 11 00 3f 00 f7 fa 28 a2 |..........?...(.|
00000210 80 3f ff d9 |.?..|
00000214
You could open it in a hex editor and start hacking away, but let’s see what FFglitch has to say about it.
1 - Export data from media file
With FFglitch, you have to choose which feature of the codec you want to glitch. This is codec-specific, and you can see more about it in the What? page.
In this example, we’ll work on the quantized DCT coefficients. More specifically, only the DC coefficients (feature name: q_dc).
We’ll export this feature from empty.jpeg into a JSON file.
$ ffedit empty.jpeg -f q_dc -e empty.json
Now open empty.json in your favorite text editor, and this is what you’ll see:
{
"ffedit_version":"ffglitch-0.7.001",
"filename":"empty.jpeg",
"sha1sum":"620ffe3810380fcd07e19a722fef8cf96a751eb3",
"features":[
"q_dc"
],
"streams":[
{
"frames":[
{
"pkt_pos":0,
"pts":0,
"dts":0,
"q_dc":{
"data":[
[
[ 127, 0 ],
[ 0, 0 ]
],
[
[ 0 ]
],
[
[ 0 ]
]
],
"luma_max":2047,
"chroma_max":2047
}
}
]
}
]
}
There’s a lot of stuff there, but you just need to focus on the “data” part:
"data":[
[
[ 127, 0 ],
[ 0, 0 ]
],
[
[ 0 ]
],
[
[ 0 ]
]
],
So we have one pair of [ ] that encompasses all the data, which consists of three pairs of [ ]. The first one contains two lines, with two numbers inside each one of them. The second and the third one contain just one line each, with just one number inside each of them.
To understand what this means, you have to understand what a macroblock is and the pixel format used by this JPEG file (yuv420p). But I will not explain this in this post :). Instead I’ll just comment on the data.
"data":[
[ ; "Y plane (contains four 8x8 blocks)"
[ 127, 0 ],
[ 0, 0 ]
],
[ ; "U plane (contains one 8x8 block)"
[ 0 ]
],
[ ; "V plane (contains one 8x8 block)"
[ 0 ]
]
],
2 - Glitch data however you see fit
We’ll start out by just experimenting on changing the values in the data structure and seeing what gives in the output. For example, let’s change that 127 to a -128. We’ll end up with:
{
"ffedit_version":"ffglitch-0.7.001",
"filename":"empty.jpeg",
"sha1sum":"620ffe3810380fcd07e19a722fef8cf96a751eb3",
"features":[
"q_dc"
],
"streams":[
{
"frames":[
{
"pkt_pos":0,
"pts":0,
"dts":0,
"q_dc":{
"data":[
[
[ -128, 0 ],
[ 0, 0 ]
],
[
[ 0 ]
],
[
[ 0 ]
]
],
"luma_max":2047,
"chroma_max":2047
}
}
]
}
]
}
Now save the file with another filename (glitch.json), and let’s see what this changed in the JPEG file.
3 - Apply glitched data and generate a new file
Now we have to tell FFglitch to apply the changes we did in the JSON file back to the JPEG file. Actually, we won’t overwrite the original file (empty.jpeg). Instead, we’ll create a new file, but ask FFglitch to get the modified values for the DC coefficients from the glitched JSON file (glitch.json). We’ll call this new file output1.jpeg:
$ ffedit empty.jpeg -f q_dc -a empty.json output1.jpeg
Now watch in awe what FFglitch did to our 16x16 pixels white JPEG canvas:
16x16 pixels no-longer-white JPEG (not to scale)
If you take a hex dump of the new file, you’ll get this:
$ hexdump -C output1.jpeg
00000000 ff d8 ff db 00 43 00 08 04 04 04 04 04 05 05 05 |.....C..........|
00000010 05 05 05 06 06 06 06 06 06 06 06 06 06 06 06 06 |................|
00000020 07 07 07 08 08 08 07 07 07 06 06 07 07 08 08 08 |................|
00000030 08 09 09 09 08 08 08 08 09 09 0a 0a 0a 0c 0c 0b |................|
00000040 0b 0e 0e 0e 11 11 14 ff c4 01 a2 00 00 01 05 01 |................|
00000050 01 01 01 01 01 00 00 00 00 00 00 00 00 01 02 03 |................|
00000060 04 05 06 07 08 09 0a 0b 01 00 03 01 01 01 01 01 |................|
00000070 01 01 01 01 00 00 00 00 00 00 01 02 03 04 05 06 |................|
00000080 07 08 09 0a 0b 10 00 02 01 03 03 02 04 03 05 05 |................|
00000090 04 04 00 00 01 7d 01 02 03 00 04 11 05 12 21 31 |.....}........!1|
000000a0 41 06 13 51 61 07 22 71 14 32 81 91 a1 08 23 42 |A..Qa."q.2....#B|
000000b0 b1 c1 15 52 d1 f0 24 33 62 72 82 09 0a 16 17 18 |...R..$3br......|
000000c0 19 1a 25 26 27 28 29 2a 34 35 36 37 38 39 3a 43 |..%&'()*456789:C|
000000d0 44 45 46 47 48 49 4a 53 54 55 56 57 58 59 5a 63 |DEFGHIJSTUVWXYZc|
000000e0 64 65 66 67 68 69 6a 73 74 75 76 77 78 79 7a 83 |defghijstuvwxyz.|
000000f0 84 85 86 87 88 89 8a 92 93 94 95 96 97 98 99 9a |................|
00000100 a2 a3 a4 a5 a6 a7 a8 a9 aa b2 b3 b4 b5 b6 b7 b8 |................|
00000110 b9 ba c2 c3 c4 c5 c6 c7 c8 c9 ca d2 d3 d4 d5 d6 |................|
00000120 d7 d8 d9 da e1 e2 e3 e4 e5 e6 e7 e8 e9 ea f1 f2 |................|
00000130 f3 f4 f5 f6 f7 f8 f9 fa 11 00 02 01 02 04 04 03 |................|
00000140 04 07 05 04 04 00 01 02 77 00 01 02 03 11 04 05 |........w.......|
00000150 21 31 06 12 41 51 07 61 71 13 22 32 81 08 14 42 |!1..AQ.aq."2...B|
00000160 91 a1 b1 c1 09 23 33 52 f0 15 62 72 d1 0a 16 24 |.....#3R..br...$|
00000170 34 e1 25 f1 17 18 19 1a 26 27 28 29 2a 35 36 37 |4.%.....&'()*567|
00000180 38 39 3a 43 44 45 46 47 48 49 4a 53 54 55 56 57 |89:CDEFGHIJSTUVW|
00000190 58 59 5a 63 64 65 66 67 68 69 6a 73 74 75 76 77 |XYZcdefghijstuvw|
000001a0 78 79 7a 82 83 84 85 86 87 88 89 8a 92 93 94 95 |xyz.............|
000001b0 96 97 98 99 9a a2 a3 a4 a5 a6 a7 a8 a9 aa b2 b3 |................|
000001c0 b4 b5 b6 b7 b8 b9 ba c2 c3 c4 c5 c6 c7 c8 c9 ca |................|
000001d0 d2 d3 d4 d5 d6 d7 d8 d9 da e2 e3 e4 e5 e6 e7 e8 |................|
000001e0 e9 ea f2 f3 f4 f5 f6 f7 f8 f9 fa ff c0 00 11 08 |................|
000001f0 00 10 00 10 03 01 22 00 02 11 00 03 11 00 ff da |......".........|
00000200 00 0c 03 01 00 02 11 03 11 00 3f 00 f9 fe 8a 28 |..........?....(|
00000210 a0 0f ff d9 |....|
00000214
And if you look at the differences between the original file and the glitched file, you’ll get this:
000001d0 d2 d3 d4 d5 d6 d7 d8 d9 da e2 e3 e4 e5 e6 e7 e8 |................|
000001e0 e9 ea f2 f3 f4 f5 f6 f7 f8 f9 fa ff c0 00 11 08 |................|
000001f0 00 10 00 10 03 01 22 00 02 11 00 03 11 00 ff da |......".........|
-00000200 00 0c 03 01 00 02 11 03 11 00 3f 00 f7 fa 28 a2 |..........?...(.|
-00000210 80 3f ff d9 |.?..|
+00000200 00 0c 03 01 00 02 11 03 11 00 3f 00 f9 fe 8a 28 |..........?....(|
+00000210 a0 0f ff d9 |....|
00000214
Notice the values that have changed at the end of the file:
- [...] f7 fa 28 a2 80 3f [...]
+ [...] f9 fe 8a 28 a0 0f [...]
This means “change white to black”, but it would have been hard to get the bitstream change right if it had been done with a hex editor…
Conclusion
We could have just used the Bucket tool in Microsoft Paint to achieve the same effect. But it’s more fun with FFglitch.