While it may not be quite clear now, documenting this will be helpful in the future. It will be a time saver when reversing firmware files given that you’ll have a clear memory map, and will provide the necessary background when thinking about exploitation.
We’ll go through each specific location one by one in the next sections.
During the boot sequence, eCos clears the .bss section. This action is executed by the hal_zero_bss function. That function is defined in ./packages/hal/mips/arch/v2_0/src/vectors.S, in pure MIPS assembly.
The function is reproduced below:
To us, the most important bits of information from the assembly are that the code loads the start address of .bss (__bss_start) into register $a0 and the end address of .bss (__bss_end) into register $a1.
Let’s look at actual firmwares to see what it looks like. On a Netgear firmware, we see that it starts with:
So for that firmware, we know that __bss_start is set to 0x816168c8 and __bss_end to 0x81b52570.
Another example, this time with an ASKEY firmware:
Here, __bss_start is equal to 0x819760b8 and __bss_end is equal to 0x81bc7660.
We discovered that hal_zero_bss always starts at the same offset (0x80004854), regardless of the firmware vendor. This is due to the way eCos compilation works and the fact that hal_zero_bss is defined before eCos packages or external libraries.
Given an arbitrary firmware file, we should be able to auto-identify the start and end locations of the .bss section by seeking to that offset and matching on the instructions setting registers $a0 and $a1.
We developed the Python 3 script below to do so:
To confirm our assumption, we ran the code on 7 other firmwares (on top of the Netgear and ASKEY ones). We successfully identified the .bss start and end address on each of them.
From cursory analysis of multiple eCos based Broadcom firmwares, we identified that the data section always starts with the string “bcm0”. Given that the .data section is at the end of the firmware file, it ends with a large amount of null bytes.
Here is the start of the .data section of an ASKEY firmware:
Here is the start of the .data section of a Netgear firmware, manually defined:
We can therefore identify the beginning and end of .data section with a script like the one below:
Similarly to the .bss section identification, we ran the script on a bunch of other firmware files and obtained valid results.
Identifying Stack Location
We initially identified the stack start address by executing this command from the CM shell of a live device:
The first task is tStartup and its dedicated stack zone starts at 0x81753c48, which is the lowest address of the list.
We obtain similar results with the command ‘stackShow’:
tStartup is always the first thread to be created on the Broadcom platform. Therefore, this thread’s stack base address will be the system’s stack base address.
The launch of tStartup is performed by calling cyg_thread_create, the assembly is provided below:
cyg_thread_create signature follows:
Instead of using registers $a0 to $a3 for parameters, and then the stack to store subsequent parameters, the Broadcom platform use an interesting convention which consists of putting parameters in registers $a0 to $a3, then $t3, $t1, $t2, and $t0.
Given cyg_thread_create signature, we’re interested in the value put in register $t3 which corresponds to the pointer to the base of the stack (stack_base), along with the value put in register $a3, which is a pointer to a string holding the thread’s name (“tStartup”).
We can auto-identify the stack start address of any Broadcom firmware by following these steps:
identifying the string “tStartup” in the binary
cross-reference that string to a location where it is loaded into register $a3
from there, match instructions setting register $t3 value. That value is the stack start address.
Identifying Heap Location
We can obtain information about the heap by going into HeapManager menu and typing stats:
It may not be obvious, but the heap start address (0x81b52570) is precisely the address where the .bss section ends.
What’s left is to identify where the heap ends. The initial heap size is 104528528 size, it’s close enough to 100MB (104857600 bytes) so let’s just consider the heap is 100MB.
Understanding how heap allocation works on eCos will be the subject of a dedicated article.
Putting Everything Together
Now that we documented the most interesting locations in the memory, let’s put everything together to have a better understanding.
The diagram below presents the whole memory used by a running device. If I’m not mistaken, there should be a dedicated stack for interrupts just below the BSS section. This region is 4096 bytes long by default. Something I still need to look into.
Thanks to everyone on the corelan slack channel for the nice debate about high addresses/low addresses locations, and whether a stack actually “grows” or not :)
I also implemented all the auto identification procedure in a Python 3 script. It’s not relying on any decompiler or disassembler library so you will get results instantly. You can get it from the dedicated repository.
From experience, this platform does not enforce any kind of permission flags on memory segments. The whole memory is read-write-execute.
In this article, we demonstrated how to reverse engineer the memory layout of the Broadcom variant of eCos.
Understanding the exact memory layout of a target is useful for exploitation and custom code injection. Now that we can identify memory regions, we know where we can write shellcode without interfering with the running system.
As always, if you have any question feel free to contact me via Twitter or email.