I’m pretty new to JTAG/EXDI debugging in Windows. My goal is to better learn ARM architecture (on a deeper level: interrupts, exception levels, boot loader, Hypervisor, secure/nonsecure world, etc.) by trying to set up at-home arm64 JTAG debugging box that can install Windows 11. I’m wondering if anyone had done that and what hardware do you recommend? And also the cost.
I believe on the software (JTAG) front I can use OpenOCD (instead of the overpriced Lauterbach’s T32) and EXDI connection to WinDbg. (Or maybe someone can correct me.)
You should use QEMU. It has a built in simulated JTAG gdb interface that works well with the windbg EXDI extension. I believe this works on x86 QEMU too. You can run QEMU in cpu emulated mode as an ARM64 system on a x86 system, with enough performance to be usable for debugging.
Unless something has improved in the last two years, on an actual ARM64 system, OpenOCD works poorly with EXDI. One of the reasons is there is no bypass of the current memory protection, which means if you try to write a software breakpoint into code (which is read-only) it will fail. Another issue is flushing cpu caches is excruciatingly slow (like an hour on a many core cpu), and this makes any operations that examine physical memory very slow. T32 has a gdb to JTAG interface, that has some non-standard extensions used by the EXDI interface, so EXDI to Lauterbach T32 works ok (not as reliably as a normal windbg connection). T32 works at all ARM64 EL levels, and understands things like ARM64 hardware ETM trace. Lauterbach’s T32 is expensive, like thousands of dollars for the JTAG hardware and then thousands of dollars per year to keep you license updated. If you’re designing systems or chips, it’s an expense in line with other expenses you have, and does work. The price is high because the number of users is small, and the complexity of the software is high.
If you want to make the ARM64 ecosystem better, feel free to fix the deficiencies in OpenOCD with windbg/EXDI, you could use a cheap ($20) JTAG adapter on a Raspberry Pi if you want. Last I knew, Windows could be booted on a Pi 4 and I’m guessing maybe now a Pi 5 (with 8 or I believe 16 GB of ram). There also are some alternatives to OpenOCD.
QEMU is also free, and you can snapshot the system state while debugging. You can also write your own virtual devices so you can debug new hardware before it’s silicon is real. If your just exploring, QEMU is absolutely the way to go. No wires or extra gadgets required.
Thanks @Jan_Bottorff I am definitely not looking to dealing with emulation. I know the architecture well enough to pass that point. And yes T32 on its own is usable. Not many people use it with EXDI though because of the limitations that you had outlined. It’s super slow. T32 on its own could be OK for a company, but for a single developer - their licensing scheme is way too expensive.
You know I didn’t know that someone can install and boot Win11 on a RPi 5. That’s pretty cool. I need to research it more.
Can you give a model name/link to the JTAG h/w for RPi that you referred to? Any tutorial for a setup too if anyone had done that before, would be super helpful.
I’ve personally not booted Windows on the RPI 5, but have on the RPI 4. You need to get the UEFI firmware for the RPI, I think the project is on GitHub.
And what are your reasons not to use QEMU? With QEMU you can do fun things like modify QEMU to instrument the instruction set, or modify the behavior of instructions, or make the peripheral environment anything you want. I JUST debugged an issue using a complex PCIe hierarchy bug in Linux using QEMU. QEMU emulation of a ARM64 core is passibly fast, like it can boot a thin Linux OS in 30 seconds or so. This is NOT the kind of slow emulation your get from Verilog level emulator. You could also buy an ARM64 laptop (Qualcomm) and potentially run it with hardware acceleration (have not personally tried this).
I’m not trying to debug anything or to modify the behavior of instructions. I may do it later. It sounds like a fun project. For now I am looking to play with an actual h/w. Thus RPi sounds quite alluring. I see people on YouTube already tried booting Win11 on RPi5.
Have you done any JTAG debugging with RPi btw? I’m totally new to RPi, so need some guidance on setting up the JTAG, etc.
PS. I thought about buying one of those Snapdragon laptops, but the issue is that they won’t have any JTAG connectors exposed. Or may not even allow disabling secure boot without blowing the fuse. And I’m way past just the stage of tinkering with general arm64 instructions.
And yes, forgot to answer your previous point. I don’t mind helping improve EXDI software if I manage to set up/find hardware for it first.
I had personally booted Windows 11 on an RPi 4 a few years ago, and you can enable JTAG. I have a RPi 5, but have not had time to play with Windows or JTAG debugging on it. I have no idea what the state of this is today. Google or now ChatGPT likely could give a howto.