Broadcom eCos | Reversing Interrupt and Exception Handling
In this article I’ll go through the different steps I followed when trying to understand the interrupt and exception handling on eCos. I initially wanted to cover this material in the Reversing eCos Memory Layout article but I started to divert too much from actual memory mappings.
While it may not be quite clear now, documenting this will be helpful in the future. It will be a time saver when reversing firmware files given that you’ll have a clear memory map, and will provide the necessary background when thinking about persistent backdoor mechanisms, custom code injection, or even building your own eCos debugger.
Let’s go through each of these locations one by one.
Common Vector (0x80000000)
The CPU delivers all exceptions, whether synchronous faults or asynchronous interrupts, to a set of hardware defined vectors. Depending on the architecture, these may be implemented in a number of different ways.
With such a wide variety of hardware approaches, it is not possible to provide a generic mechanism for the substitution of exception vectors directly. Therefore, eCos translates all of these mechanisms in to a common approach that can be used by portable code on all platforms.
On MIPS, most exceptions and all interrupts are vectored to a single address at either 0x80000000 or 0xBFC00180. Software is responsible for reading the exception code from the CPU cause register to discover its true source. One of the exception codes in the cause register indicates an external interrupt. Additional bits in the cause register provide a first-level decode for the interrupt source, one of which represents an architecture defined timer.
The mechanism implemented is to attach to each hardware vector a short piece of trampoline code that makes an indirect jump via a table to the actual handler for the exception. This handler is called the Vector Service Routine (VSR) and the table is called the VSR table.
The trampoline code performs the absolute minimum processing necessary to identify the exception source, and jump to the VSR. The VSR is then responsible for saving the CPU state and taking the necessary actions to handle the exception or interrupt. The entry conditions for the VSR are as close to the raw hardware exception entry state as possible - although on some platforms the trampoline will have had to move or reorganize some registers to do its job.
Let’s read the content at offset 0x80000000:
By disassembling the obtained bytes, we uncover the trampoline code:
It’s a perfect match for this piece of assembly from eCos 2.0 source:
The code isolate the exception code and use it as an index for the virtual service routine table. From the trampoline code we disassembled, we know the VSR table starts at offset 0x80000300.
The hal_vsr_table is defined in assembly in ./packages/hal/mips/arch/v2_0/src/vectors.S. We can see that it has 16 entries (64 / 4).
Let’s read 64 bytes starting from offset 0x80000300:
The virtual service routine table is a table with 16 entries, each of them pointing to a specific function. By comparing the disassembly and the actual assembly from ./packages/hal/mips/arch/v2_0/src/vectors.S, I was able to precisely identify the functions:
0x800043ec - __default_interrupt_vsr
0x80004bd8 - __default_exception_vsr
To fully understand how the trampoline index the VSR table, I wrote the following Python snippet:
Executing the code will give us the following output:
As we can see, the only valid exception codes that index within the VSR table are values between 0 and 15 included. These are valid exception codes that we can find in the MIPS documentation:
Cause of exception
Address Error exception (Load or instruction fetch)
Address Error exception (Store)
Instruction fetch Buss Error
Data load or store Buss Error
Reversed Instruction exception
Arithmetic Overflow exception
Floating Point Exception
Please note that item 1, 2, 3, and 15 are undocumented.
An interesting fact is that, due to the way eCos firmwares are compiled and assembled, the location of __default_interrupt_vsr and __default_exception_vsr is the same for all firmwares based on the Broadcom variant of eCos.
The following piece of code takes advantage of that fact and gather the VSR information from a live system over a serial connection. You can find the code in the dedicated repo.
Running the code against a Netgear CG3700B device:
The visual representation below should help you understand how all these different components interact with each other:
An annotated version of __default_interrupt_vsr assembly is provided below to help you dig even more into the subject.
MIPS exceptions are handled by a peripheral device to the CPU called coprocessor 0 (cp0). Coprocessor 0 contains a number of registers used to configure exception handling and to report the status of current exceptions.
An annotated version of __default_exception_vsr assembly is provided below to help you dig even more into the subject.
Stub Entry Vector (0x80000100)
The stub entry vector is supposedly located at 0x80000100, so let’s read 32 bytes from starting from there.
The disassembly is exactly the same than the common vector:
It’s a perfect match for this piece of assembly from eCos 2.0 source:
Similarly, this trampoline will fetch an address from the VSR table and jump to it.
Debug Vector (0x80000200)
Debug vectors are not used in production system, but let’s document it for completeness sake.
Let’s read the first 32 bytes starting at offset 0x80000200 and disassemble them with rasm2.
This is similar to the assembly, although a little more convoluted:
Virtual Vector Table (0x80000400)
“Virtual vectors” is the name of a table located at a static location in the target address space. This table contains 64 vectors that point to service functions or data. The fact that the vectors are always placed at the same location in the address space means that both ROM and RAM startup configurations can access these and thus the services pointed to. The primary goal is to allow services to be provided by ROM configurations (ROM monitors such as RedBoot in particular) with clients in RAM configurations being able to use these services. Without the table of pointers this would be impossible since the ROM and RAM applications would be linked separately - in effect having separate name spaces - preventing direct references from one to the other. This decoupling of service from client is needed by RedBoot, allowing among other things debugging of applications which do not contain debugging client code (stubs).
This value contains the total number of virtual vectors in the upper 16 bits, and the definition number of the last supported virtual vectors in the lower 16 bits. For this VVT, the total number of virtual vectors is 64d (0x40), and the definition number of the last virtual vector, Flash ROM Configuration, is 20d (0x14). The version is therefore 0x4014 (edit: actually 0x00400014).
Here, the virtual vector table version is set to 0x00080015.
The definition number of the last vector, Flash ROM Configuration is 21d (0x15). The total number of virtual vectors in the upper 16 bits should be 64d (0x40), but is actually 8d (0x8). It’s highly probable that Broadcom changed the initial eCos behavior. We can still rely on the lower 16 bits though.
To interact with the VVT, I wrote a piece of Python code that lists the entries from a live system by fetching the information over serial.
If you made it through here, congratulations ! Our acquired understanding of the inner workings of eCos interrupt/exception handling and dedicated vector tables will be helpful in the future when we try to inject GDB stubs into running production firmware. This will also prove useful when we will be designing backdoor persistence by hijacking vector table entries.
As always, if you have any question feel free to contact me via Twitter or email.