NAND Flash Memory
read this
http://spritesmods.com/?art=ftdinand&page=1
http://www2.lauterbach.com/pdf/nandflash.pdf
https://www.kernel.org/doc/menuconfig/drivers-mtd-Kconfig.html
http://www.informit.com/articles/article.aspx?p=1187102&seqNum=5
http://www.yaffs.net/
http://www.stlinux.com/howto/NAND
From http://h30499.www3.hp.com
Reverse Engineering NAND Flash Memory – POS device case study (part 1/3)
Modern life wouldn’t be quite so modern without NAND Flash memory. It is used everywhere – from USB sticks and smart phones to security cameras and the Internet of Things (IoT). I’ve spent some time reverse engineering the NAND flash memory used in popular POS (Point of Sale) devices. (Figure 1) Aside from sharing my findings from the evaluation of these devices, I wanted to share the process I followed to acquire the right firmware and to perform the reverse engineering. You can find a brief introduction to this area in one of my previous blogs. I also recently presented the details of my research at Recon and Black Hat 2014. While the Recon presentation focused on the technical details of the technique, at Black Hat 2014, I showcased an advanced tool called DumpFlash and shared my experience with tamper detection, including a workaround method. In this series of blogs, I’m going to share these details and some other interesting facts as well.
Figure 1 Targeted device
The first step when reverse engineering an embedded device is to open it up and identify the chipsets, looking for any interesting points that might be JTAG or UART. Actually, it turns out that the chipsets are really standard with this POS device. (Figure 2) It uses the very popular ARM CPU (s3c2410 series) with a typical setup with DRAMs and NAND Flash memory. However, the attempt to find JTAG or UART didn’t work very well for me. Instead of struggling with something not so obvious, I took a very obvious approach – dumping NAND Flash memory and programming it directly.
Figure 2 Inside of targeted device
The basic pinout is shown in Figure 3 and this device works in 8bit mode. There are Flash memories that work in 16bit mode, but they have a slightly different pinout configuration.
Figure 3 NAND Flash memory pinout (x8 device)
De-soldering
The first step for directly interacting with NAND Flash memory is de-soldering it from the PCB board. But, there is another very tempting option you can take using a special clip that can be attached to the surface of the NAND Flash memory. You can find this clip in stores that stock XBOX-modding tools – it was originally used for modding XBOX 360 machines by patching them directly. However, the problem here is that this option doesn’t always work. Note that there are pins for supplying power to the NAND Flash memory and the chip remains connected to the PCB board with this method. Unfortunately using the clip in this situation can lead to current injection into the board, which wakes up other chipsets and interferes with your operations. That is certainly what happened to me when I tried this clip. But I know of several cases where this clip was used successfully, so definitely in some cases, it looks like the clip works.
Figure 4 Failed attempt – want to avoid de-soldering?
After I decided to de-solder the NAND Flash memory, the first method I tried was using a de-solder wire. This method is valid, but the problem I had was that there was too much residue left on the board and the chip. I was going to re-solder the chip back after I had extracted the firmware and re-programed it. I wanted the PCB board and the chip in cleaner condition, so I dropped this approach.
Figure 5 Failed attempt – want to use de-solder wire?
The most effective way to remove the chips cleanly was using a hot air gun from an SMT rework station. (Figure 6) Insulation tape is good for protecting the surrounding areas on the targeted PCB.
Figure 6 SMT rework station & insulation tape
I prefer applying insulation tape over the area around the targeted chip.
Figure 7 Apply heat over the pin area
If you apply too much heat over an extended period, it will eventually damage the chip. (Figure 8) You might still try to read the chip, but if the chip is damaged, it may not be recognized at all or it might just return a series of 00 bytes when you try to read data from it.
Figure 8 Too much heat?
The following video demonstrates the detailed process of removing a chip from the PCB board using a hot air gun.
Reading NAND Flash Memory
Now you have a bare NAND Flash chip at hand. The next step is reading the bare metal image from the chip. There have been a lot of different approaches tried over time by the hobbyist community –some use special Flash reader chipsets that can provide low-level access. I tried it and it didn’t work quite as well as expected. (Figure 9)
Figure 9 Failed attempt – modding an SD reader
The most reliable way I found to do this was bit-banging using the FTDI FT2232H chip set. This method was originally suggested by Sprite Mod. Bit-banging is a technique that allows you to directly interact with chips through software. FT2232H is a versatile chip that provides various ways to interact with chips through a USB interface.
Figure 10 FT2231H chip and breakout board
Figure 11 shows the connection between the FT2232H chip and NAND Flash memory. The connections are mostly based on information from Sprite Mod, but there is a slight modification between the BDBUS6 and the CE (9) connection.
Figure 11 Connection between FT2232H and NAND Flash Memory
A detailed explanation about what each connection means is provided with my Black Hat 2014 white paper.
MCU Host Bus Emulation Mode
FTDI FT2232H supports multiple modes. The ‘MCU Host Bus Emulation Mode’ is appropriate for our purposes in this case. In this mode, the FTDI chip emulates an 8048/8051 MCU host bus. By sending the commands shown in Table 1 and retrieving the results, the software can read or write bits through I/O lines. More details are available in a note published by FTDI.
| Commands | Operation | Address |
| 0x90 | Read | 8bit address |
| 0x91 | Read | 16bit address |
| 0x92 | Write | 8bit address |
| 0x93 | Write | 16bit address |
| 0x82 | Set | High byte (BDBUS6, 7) |
| 0x83 | Read | High byte (BDBUS6, 7) |
Table 1 FT2232H commands
NAND Flash Memory Commands
Table 2 shows the basic command sets usually used by NAND Flash memory. There are more complicated commands available depending on the chipsets, but these basic commands are enough for essential operations like reading and writing data on the chip. Also, these commands tend to be the same across different models and brands. The pins and other descriptions presented here are mostly focused on small block NAND Flash models (512 bytes of data with 16 bytes OOB). Models with a large block size use a different set of commands, but the principle is the same.
| Function | 1st cycle | 2nd cycle |
| Read 1 | 00h/01h | – |
| Read 2 | 50h | – |
| Read ID | 90h | – |
| Page Program | 80h | 10h |
| Block Erase | 60h | D0h |
| Read Status | 70h |
Table 2 Basic command sets for usual NAND Flash memory (small blocks)
Building Reader/Writer Device
Figure 12 shows the final NAND Flash reader/writer assembled based on the connection information shown in Table 2. You can make a device like this even with a relatively low budget and basic knowledge of electronics engineering. You need an FTDI FT2232H breakout board, a USB cable, a TSOP48 socket, and wires. Everything can be constructed with budget of USD $50.
Figure 12 NAND Flash reader/writer
Place your NAND Flash chip inside the TSOP48 socket. (Figure 13) This socket is very useful as you can safely place your NAND Flash chip inside it and then directly interact with the extended pins without touching and possibly damaging any Flash memory chip pins.
Figure 13 TSOP48 socket
When the NAND reader/writer is ready, just load the Flash memory. You should be careful to put the pin 1 location on the correct side of the socket. Usually the socket shows where pin 1 should be located. (Figure 14) When things are set, you can connect the reader/writer to the computer through a USB cable.
Figure 14 Pin 1 location
DumpFlash – enhanced Flash reader/writer software
You need software to achieve bit-banging and for this there is a NANDTool open source project maintained by Bjoern Kerlers. I ported the entire C++ code to a Python project and created the DumpFlash project. I also added support for NAND Flash programming. Here are the highlights:
Read/Write support
- Flash image manipulation and verification (ECC, Bad block check)
- Fast sequential row read mode support for Samsung models
- Automatic U-Boot image extraction
- JFFS2 parsing and extraction
- Support for enhanced schematics
Download the DumpFlash code first. You should also install prerequisite packages like pyftdi and libusbx. With everything set up, you can query basic Flash information using the –i option. (Figure 15)
Figure 15 Retrieving Flash memory information (-i)
With the –r option, you can read contents from the chip. The average transfer rate I usually get with my Windows 7 machine is around 50KB/sec.
Figure 16 Reading Flash memory data (-r)
The FlashTool also supports sequential row read mode. When you can specify the –s option, it uses the special mode with a Samsung chip and increases reading performance. The reading speed is faster than normal page-by-page mode by 5-6 times in this case. (Figure 17)
Figure 17 Reading Flash memory data in sequential mode(-r -s)
Conclusion
With the schematics and software I’ve provided, you have the ability to read and program NAND Flash memory. De-soldering is not a big problem, even if you are a hardware novice. You just need an SMT rework station and a little bit of a courage. The real challenge happens later when you try to dissect the firmware, put it back together and re-solder the chip back to the PCB board. However, often the purpose of firmware acquisition is to do white-box testing, so with acquired firmware, you can have a look inside the internal system and use that knowledge to pen-test the device (and that’s probably the reason why the NAND-related open source project didn’t have code for programming data back to NAND Flash memory).
Now you are ready to do something useful in an embedded system, but what’s next? In the next post, I’ll share a method for dissecting a flat bare-metal image, and throw in a few helpful tips and strategies for good measure. After that, I’ll talk about modifying firmware and putting it back into the machine again. This involves some special maneuvers with a POS device. The model I worked on featured so-called hardware tamper detection and I’ll talk about how to work around that using a software method.
Reverse engineering NAND Flash Memory – POS device case study (part 2/3)
In my previous blog, I talked about a method for acquiring bare-metal images directly from NAND Flash memory. In that post, I used a Verifone POS device as my reverse engineering example. The next step is to dissect the image and change it into a more meaningful form for our investigation. Let’s start with making sense of the out-of-band (OOB) data from the image and removing bad blocks. After that, we will have a flat image that can be used for further analysis.
OOB data
The OOB data is used for saving Error Control Code (ECC) and other metadata like bad block markers. The chip I worked on has a 512 byte page size and 16 bytes of OOB data.
Figure 1 Page + OOB
ECC
ECC is used for detecting errors in the data inside a page. The concept of ECC is also used in hard disk technology and is based on Hamming code. Hamming code is older technology that can be traced back to the 1950s when Richard Hamming invented it for reducing repeated work for punch cards. Even when the Hamming code algorithm is very straightforward, different vendors may use different ECC variation algorithms. I’m going to explain the ECC algorithm used in the target chipsets I worked on.
Figure 2 shows a graphical representation of an ECC calculation table. Each cell marked “Bit0” to “Bit7” represent the bits that make up each byte in a page. In this case, I’m talking about a page with 512 bytes. Basically, we need to calculate row parity bits (P8, P8’, P16, P16’, P32, P32’, P64, P64’, P128, P128’, P256, P256’, P512, P512’, P1024, P1024’, P2048, P2048’) and column parity bits (P1, P1’, P2, P2’, P4, P4’).
Figure 2 ECC calculation table
Figure 3 and Figure 4 show how row parity bits are calculated. For calculating P8’, it uses bits from every even byte (byte[0], byte[2], byte[4], …) to calculate the parity bit.
Figure 3 Row checksum calculation (P8’)
As another example, P16’ uses two bytes and skips two bytes, then continues this until the end of the bytes. A similar principle can be applied to other row parity bit calculations.
Figure 4 Row checksum calculation (P16′)
Figure 5 shows a proof of concept code written in Python that displays row parity checksum calculations. The xor_bit is the parity bit calculated for each byte. The i variable is the row number, starting from 0 to 511.
Figure 5 Code for calculating row checksums
After you calculate all the row parity checksums, you calculate the column parity checksums. Column parity checksums are calculated using the same bits from all bytes in the page. For example, to calculate the P2 checksum, you choose bits 2, 3, 6 and 7 out of each byte. (Figure 6)
Figure 6 Column checksum calculation
Figure 7 shows proof of concept code that illustrates how these row parity checksums are calculated.
Figure 7 Row checksum calculation code
Now when we have calculated these checksums, we need to consolidate these bit values to bytes. The method used for column parity checksums and row parity checksums is usually common to different NAND Flash chipsets. However, the method that expresses how these bits are consolidated can differ. The method used for the specific chipset I worked on is shown in Figure 8.
Figure 8 ECC calculation code
How these ECC bytes detect a one-bit error is pretty obvious. A one-bit error triggers parity errors for multiple column and row checksums. By checking which checksums are wrong compared to OOB ECC values, you can pinpoint the exact bit that triggered the checksum error. As a bit has only two states (0 and 1) if a value is wrong, the other one must be correct.
Bad blocks
ECC can verify the integrity of each page. It can even find and fix minor bit errors. But, if there are too many bit errors in a page, the whole block may be marked as a bad block. Usually the first or last page of a block is used for marking bad blocks. For the chipset I worked on, based on the specification, the 6th byte of OOB bytes is used for marking bad blocks. If the value is not 0xFF, it indicates a bad block.
Figure 9 Bad block marking
In the real world, bad blocks are very common. Even in new NAND Flash memory fresh from the factory, there can be factory-marked bad blocks. Instead of throwing away chips with minor defects, they mark bad blocks in the factory. Usually, these manufacturers guarantee that the percentage of bad blocks doesn’t go over a particular level.
The DumpFlash tool supports scanning for bad blocks. Figure 9 shows that this chip has 2 bad blocks.
Figure 10 Bad block scanning using DumpFlash
Reverse engineering Flash memory data
Now you have acquired a flat firmware image for further analysis. The next step is identifying the layout of the firmware image. A firmware image is usually composed of multiple parts with different functions. On the machine that I worked on, the firmware layout looks like that shown in Figure 11.
Figure 11 An example of Flash memory layout
The first stage bootloader is the code loaded by the ARM CPU when it first boots up. It performs a very low level initialization of the system and eventually loads the next level bootloader. There are several different kinds of second level bootloader and the U-Boot bootloader is one of the most popular. The U-Boot bootloader here is the binary code that actually boots up the OS kernel.
The target machine I worked on had two U-Boot images:
- U-Boot image 1 had the ramdisk image with the Linux kernel
- U-Boot image 2 had the Linux kernel image
The ramdisk image from the U-Boot image 1 has a recovery file system that contains essential tools that can be used to recover a damaged system. This image is only loaded when the bootloader detects something wrong with the main image. Under normal circumstances, the Linux kernel image from image 2 is loaded. After the kernel boots up, the main file system from the JFFS2 file system is loaded to memory and the system initialization scripts run.
One challenge here is how to identify this layout more efficiently. There are several different approaches you can take. You can use the binwalk tool to identify popular image patterns (it recognizes U-Boot images easily) or you could use one of the three simple approaches presented below.
Graphical analysis
When NAND Flash is first manufactured in the factory, it is in an erased state, which has all the 0xFF values filled in. The fact that the space is filled with 0xFF is really helpful when identifying the firmware layout. When an engineer first designs the layout of the firmware, they allocate enough space for each image. For example, when you compile a Linux kernel, depending on the features you put into the kernel, the kernel sizes can be drastically different. So an engineer would assign reasonable enough space for a U-Boot image space that holds that kernel file. But, you don’t always fill up the whole space. This makes a hole between images. Figure 12 shows the actual BMP image generated from the firmware image I worked on. I used bin2bmp to convert the firmware file to a BMP image. I used the following command to generate the image:
c:\python27\python bin2bmp-0.1.6.py -w 4096 Image. dmp
The –w option specifies the width of the image, which is 4096 pixels. As 4 bytes makes one pixel, 4096 pixels represents 16384 bytes – which is the size of one block in this case. So one row is one block in the image. As there are 4096 blocks in the firmware image, there is the same number of rows in the BMP file.
Figure 12 Graphical representation of the bytes in the firmware
When you look at Figure 12, you can easily identify the compartments. The first stage boot loader usually takes only one block and it’s barely discernable in this picture. But, the boundaries of the U-Boot binary, other U-Boot images and the JFFS2 image can be seen clearly here. Even inside the JFFS image, you can see that it is split into multiple parts. I guess this is related to some wear leveling action from the JFFS2 file system.
Figure 13 Graphical representation of the bytes in the firmware
To look closely, I zoomed into the area in the red square in Figure 13. The zoomed-in image is shown in Figure 14.
Figure 14 Graphical representation of the bytes in the firmware (zoomed in)
One notable pattern you can see in the picture is the JFFS2 header. Have a look at the yellow box in Figure 14 and you can see some repeating patterns occurring. These are the JFFS2 header patterns. The JFFS2 header has 2 bytes of magic value and other fields following them. (Figure 15) The magic value is always the same, but the node type and total length changes. The node type value only has a set of a few different values and the total length is not always totally random, as one JFFS2 record usually fits into one page. These facts make the JFFS2 patterns very noticeable. This can be used as another hint for identifying JFFS2 patterns graphically.
Figure 15 JFFS2 Header fields
Pattern-based analysis
Graphical analysis is very intuitive and useful, but you can also use a pattern-based approach. Each compartment starts with some specific patterns. For example, U-Boot images have a U-Boot image header structure at the start of the image and the image always begins at the start of a block. This makes identifying U-Boot images very easy. Just iterate each block, read the first few bytes and find the U-Boot image header pattern. The U-Boot image header has a 4 bytes magic value of 0x56190527, so it is relatively easy to pinpoint U-Boot images. The other advantage to using this method is that the header information can also be used to determine the size of the U-Boot image itself.
Figure 16 U-Boot header structure
Figure 17 U-Boot header example
Aside from the U-Boot images, as I mentioned, the JFFS2 image also has specific header patterns and it is possible to easily identify the start and end of the JFFS2 image. But, for the JFFS2 file system, there is one more indicator that can be used to determine the image location. When the JFFS2 file system first uses a specific Flash memory area, it formats the region using an erasermarker. This marker appears on the first page of the block. When you search for the row firmware image that includes the OOB area, this erasermarker pattern can be used to determine the JFFS2 file system. (Figure 18)
Figure 18 JFFS2 erase marker location from a page and spare column bytes
Bootloader environment variable analysis
The last method uses U-Boot environment variables. This method applies when you are analyzing firmware that uses a U-Boot bootloader. Locating the U-Boot bootloader is trivial as it usually comes after the first stage bootloader. With my target machine, the U-Boot bootloader started at the second block of the chip. The U-Boot bootloader is a very versatile bootloader and can perform various actions. One such feature is creating an environment variable area and using it as the boot environment variables when the system boots up. The environment variables can be set and unset using interactive bootloader prompt commands. Depending on the configuration, you might set it to use a hardcoded default boot configuration. Figure 19 shows the code that uses default environment variables. The variables are defined with the build setup and are hardcoded in the binary.
Figure 19 U-Boot code showing default_environment
Figure 20 shows the part of the U-Boot disassembly that loads up the default_environment variables.
Figure 20 Loading default_environment variables
The following shows one of the lines that defines OS boot-up arguments:
bootargs=root=/dev/mtdblock3 rootfstype=jffs2 noinitrd ramdisk_size=4096 mem=32M mtdparts=s3c2410-nand:16k(boot),176k(u-boot),4912k
This bootargs shows a few different facts about the layout of the image. First, rootfstype=jffs2 shows the main file system of the system. The mtdparts variable actually shows the layout of the images inside the firmware. (Table 1)
| mtdparts string | Name | Blocks |
| 16k(boot) | 1st stage bootloader | Block 1 (1 block) |
| 176k(u-boot) | U-Boot code | Block 2 – 12 (11 blocks) |
| 4912k(linux-img) | U-Boot images | Block 13 – 319 (307 blocks) |
| 27104K(rootfs) | JFFS2 file system | Block 320 – 2013 (1694 blocks) |
Table 1 The bootargs mtdparts string
Using the mtdparts variable, you can locate the firmware layout very accurately. Moreover, the following bootcmd and altbootcmd commands can be used to find the exact location of the main and alternative U-Boot images:
bootcmd=nand read.j $loadaddr $nandoff_image $nandsize_image; bootm
loadaddr=0x30A00000
nandoff_image=0x330000
nandsize_image=0x136001
altbootcmd=nand read.j $loadaddr $nandoff_alt_image $nandsize_alt_image; bootm
nandoff_alt_image=0x30000
nandsize_alt_image=0x300000
For example, the main boot image location nandoff_image=0x330000, shows the exact start location of the U-Boot kernel image for boot-up. The nandoff_alt_image=0x30000 option shows the recovery U-Boot image location. Now, just using the U-Boot environment variables, you can identify the layout of the Flash memory easily.
Conclusion
So with this part two blog, I talked about OOB data and how we can use its information to verify and fix a firmware image. I also showed various methods for discovering its layout. The next step in our journey with this firmware analysis is analyzing the JFFS2 file system and extracting interesting stuff out of it. After that, we might even modify the file system and put the extracted NAND Flash memory back into the board to use it for further research. That will be our main objective in part three – coming soon.
Reverse engineering NAND Flash Memory – POS device case study (part 3/3)
In my first blog, I talked about a method for acquiring a NAND Flash memory image by directly interacting with the chip. After you acquire a raw firmware image, using the various approaches I proposed with my second blog, you should be able to easily identify the layout of the firmware. At this point in the process, it’s time to extract the data and manipulate it.
Figure 1 shows a typical firmware image layout. (This is also relevant to the POS device I worked on.) The U-Boot bootloader can be replaced with another bootloader if you like, and the JFFS2 file system can also be replaced with another popular journaling file system.
Figure 1 Typical firmware image layout
1st stage bootloader
The 1st stage bootloader is usually very architecture-dependent code that performs hardware initialization. The code is automatically loaded at address 0x00000000 by an ARM CPU when it powers up. Figure 2 shows the hardware initialization code from the image I worked on.
Figure 2 Hardware initialization code
The 1st stage bootloader usually loads up the 2nd stage bootloader – like a U-Boot bootloader for example. This is more of a general purpose bootloader and has more features than the 1st stage bootloader. This 2nd stage bootloader enables user interaction through UART, etc. It can also identify and load the kernel, and various images from various sources – including NAND Flash memory and the network location.
Figure 3 Code from U-Boot bootloader
U-Boot and U-Boot images
The U-Boot bootloader uses U-Boot images to pack the next level OS and recovery file system images. Figure 4 shows the definition of the U-Boot image header. It has a magic value of 4 bytes (27 05 19 56) and a new image always starts with a new block. So identifying U-Boot image is relatively easy. Figure 5 shows a typical U-Boot image header.
Figure 4 U-Boot image header
Figure 5 U-Boot image example
Sometimes, one U-Boot image contains multiple sub-images – Figure 6 shows an example. It contains a Linux kernel image and a Ramdisk image. This is a recovery OS image that is used when the main file system image is damaged or the main kernel doesn’t boot up correctly for some reason.
Figure 6 Multi-file image
The DumpFlash tool can be used to dump information and extract sub-images from U-Boot images. Figure 7 shows a good example when using the –U option to extract every U-Boot sub-image from the Flash device.
Figure 7 DumpFlash.py -U option to extract all sub-images
After acquiring the sub-images, depending on the type of the image, you can apply various analysis methods. If the file is a file system image, you can actually mount it on the system and browse the contents. Figure 8 shows an example of mounting an ext2 file system image file extracted from U-Boot image blocks.
Figure 8 Mounting RAM disk image
Journaling file system
Modern embedded systems usually use a journaling file system. This has some advantages over traditional file systems when initial performance is not as good. Usually when the machine starts up, it loads up the whole file system on DRAM first, with modifications to the file system being performed in memory before it is synced to Flash memory. The journaling file system makes it possible to split a file write into small chunks of records, so that it will not re-write the whole file contents when only small parts of it are modified. This is beneficial because reading and writing NAND Flash data is a slow process and excessive writing can diminish the lifespan of a NAND Flash device.
Identifying a JFFS2 file system (one of the popular journaling file systems used in the device) is relatively easy. When the JFFS2 software first prepares the NAND Flash device for JFFS2, it leaves a special marker called an erasermaker in the OOB area of the first page of each block. The erasermaker bytes are usually 85 19 03 20 08 00 00 00. (Figure 9)
Figure 9 JFFS2 erasermakrer
After the JFFS2 file system, investigating the contents is very straightforward. Just by using MTD (Memory Technology Device) in Linux systems, you can directly mount it as a file system. (Figure 10)
Figure 10 Mounting JFFS2 image
You can then browse the contents using usual Linux commands. (Figure 11)
Figure 11 JFFS2 contents
Modifying firmware
Acquiring firmware in itself is very useful for further analysis and vulnerability research. It is also beneficial if you can modify the firmware and reload it from the machine. This is not quite as easy, but it’s certainly not impossible. The first thing to do is to find the target code to patch. After this you’ll need to fix all related page checksums.
For example, the device I worked on has tamper protection. If the device is opened up at least once, it won’t boot up correctly and displays error messages similar to those shown in Figure 12. This is to alert the POS owner if any attempts to modify the device are made. Tamper protection is a big deal with POS devices because they process financial transactions. Obviously, if it is back-doored or tampered with the integrity of the system is lost and it could potentially leak credit and debit card information, such as track 1 and track 2 data.
Figure 12 Tamper protection in action
When I opened up the device, I saw another very curious device inserted between the front and back panels. This device connects the circuits of the front and back panels. (Figure 13)
Figure 13 Front and back panels
When I looked closely at the device, it appeared more interesting. (Figure 14) First I thought it was like plastic padding or something, but it actually had circuits inside it. And as I discovered, trying to tamper with this device is not easy. The conducting materials are painted inside the cover and any attempt to tamper with this easily breaks the circuits. So I suspect this device is used for tamper detection. When you open up the device, the current between the back and front panel is broken, and there might be a proprietary chip that detects this. Of course, this is just speculation for now.
Figure 14 Circuits inside plastic padding
The thing is that I don’t know exactly how tamper detection is performed at the hardware level but it is relatively easy to patch this up in the software level. To circumvent this protection, you need to find the process that actually checks the tamper detection in the device. The process that is responsible for this is the /bin/svcsec program. It is loaded with the rcS script when the system starts up. It uses a proprietary device called /dev/spectrum to retrieve tamper information. The code that is doing the tamper detection is shown in Figure 15 and you can patch the CMP instruction to the CMN instruction to change the control flow.
Figure 15 Patching CMP instruction
Now that you have a patched binary, you need to write this file into the flat JFFS2 image file. Figure 16 shows this process. DumpJFFS2.py has various command line options. The –t and –s options specify the location and size of the patch location in the file. Here we patched 4 bytes at offset 0x11380 of /bin/svcsec file. It reads the original JFFS2 dump file – named JFFS2-01.dmp – and modifies the affected JFFS2 records, writing the output to the JFFS2-01-Patched.dmp file.
Figure 16 DumpJFFS2.py tool to overwrite modified JFFS2 record
Figure 17 and Figure 18 show how this command modifies the affected JFFS2 record. The original record has a compressed data size of 0xF4 bytes. If you decompressed this data using the ZLIB library, it would be part of the executable code from /bin/svcsec. The decompressed size is 0x100 bytes. Out of these bytes, we modify only 4 bytes and compress the entire data again, which creates new compressed bytes of 0xEF size. After appending a few 0xFF padding bytes to fill the space between this and next JFFS2 record, the script calculates 3 CRC values – header, data and node – and writes them back to the header.
Figure 17 Original JFFS2 record
Figure 18 Modified JFFS2 record
Now with the patched JFFS2 raw image, you need to flash it back to Flash memory. With the changes made, it affects one page in this case. It doesn’t need to flash whole blocks and pages of the image, it just needs to write back one page. Figure 19 shows the command line options to achieve this. The –OJ option specifies the patched dump file to write back to and the –C option specifies the original JFFS2 dump file. The program compares both images and only writes the modified data. The –b option specifies the range of JFFS2 file system blocks. We already got this information with the DumpFlash.py –j command. You need to erase a block before writing NAND Flash pages – that is how it usually works with Flash memory, so the script will erase the affected block and rewrite the whole block, not just the page.
Figure 19 DumpFlash to write affected page
Movie 1 shows the process of re-soldering the Flash memory back to the board after modifying the firmware. If this process is successful, you can turn on the device and see it working again with the modified firmware.
Movie 1 Re-soldering Flash memory back to the board
Conclusion
After identifying the layout of the firmware using the various methods I shared last time, you can extract bootloader and file system images. I talked about extracting and modifying U-Boot images and JFFS2 images, which are very popular with embedded systems. Journaling file systems are especially interesting in the way they work and how they are laid out in the image file. Modifying a few bytes in the binary inside the file system involves various levels of modification to the image. First, the whole JFFS2 record that contains the affected bytes needs to be modified. Then the page that contains the specific JFFS2 record needs to be checksumed again. Next you need to write the image back to the physical Flash memory. Finally, you need to re-solder the chip to the board again. I used the example of a POS device. This POS device had tamper protection, but the check for tampering was performed in userland process level, which means that if you can modify the code, the protection is off. So even with very sophisticated hardware devices that detect tampering, a few bytes of code modification nullifies whole physical security concept.