Eureka again! I finally figured out how to use Yocto to build a Linux kernel with all of the symbolic debug information, so that I can see all of the source code and symbols within SourcePoint.
In Episode 43: Exploring the Linux Kernel with SourcePoint, I did a special Yocto build of a core-image-minimal Linux for the MinnowBoard Turbot, launched it, and then used SourcePoint to halt the boot process and look at the code. I did a fair amount of research trying to determine exactly the right settings to use in the local.conf file to build a Linux kernel with the needed symbol/source support, and thought that putting in these three lines would do it:
EXTRA_IMAGE_FEATURES ?= “debug-tweaks dbg-pkgs”
DEBUG_BUILD = “1”
SELECTED_OPTIMIZATION = “${DEBUG_OPTIMIZATION}”
But, when I JTAG-halted the target and then loaded the vmlinux file with the Load Program dialog box:
I got some, but not all, of the symbols I wanted; and I could not see any source code displayed in the Code window when I tried to switch from Disassembly mode to Mixed or Source mode:
If you look closely at the Code window above, you can see the reference to the start of the rtl8169_init_phy function at the top, a JMP and a couple of CALLs to __x86_indirect_thunk_eax, a CALL to msleep, a CALL to rtl_apply_firmware, and a CALL to rtl_eri_write. But there is also a JA (Jump If Above) to a raw address (FFFFFFFF81621FA1). The code just doesn’t look complete. And, no source code, no matter what I did. It was time to do a little more research.
After spending a few hours over the span of a couple of weeks fruitlessly Googling and trying things on my own, I gave up and did what I rarely do: I asked one of our Field Application Engineers (FAEs) for help. Recognize that I didn’t do this lightly: not only do I respect their time (they are, after all, spending most of their days helping customers with real debug problems, and not indulging an itinerant sales guy’s hobbies), but part of my joy of exploration is in figuring out solutions on my own. So, when I asked, I simply requested a “hint”: any idea why I can’t see the source and all of the symbol information?
The answer came back without giving too much away: the hint was that my vmlinux file was too small. If you look at the top diagram above, you can see that it’s only about 25MB in size. “There’s no way a kernel that small can have all of the symbols and source path information in it. It needs to be significantly larger”, I was told. So, it was up to me to figure out how to do a build with symbols; obviously, just adding a few lines to the local.conf file as I described above was not sufficient.
At first, I thought that the strategy was to adjust the global compiler flags to add support for dwarf2 symbols in the meta/conf/bitbake.conf file. I inferred this information from a couple of articles, entitled How to add global XX compiler flag to yocto build and Writing a Linux Debugger Part 4: Elves and dwarves. And I knew that SourcePoint supports the various dwarf symbol file formats. I had some familiarity with the bitbake.conf file from before: as I tinkered with the local.conf settings needed to build a debug image, I found that I had to change the default setting of the INITRAMFS_MAXSIZE inside bitbake.conf for the image, by changing it to:
INITRAMFS_MAXSIZE ??= “169318”
Otherwise, the build would error out.
Through some guesswork and experimentation, I made the following changes to the build flag option:
export TARGET_CFLAGS = “${SELECTED_OPTIMIZATION} -gdwarf-2”
Alas, this by itself did not quite work. I still got the same incomplete symbol information, and no source, in the SourcePoint Code window. In fact, the vmlinux file did not budge in size from 25MB. Was it time to throw in the towel, and go in pursuit of easier prey?
Then, out of the blue I got the following inspiration: perhaps the compiler flag was under my nose all along, in the menuconfig (see Episode 40, Hacking the Linux Kernel, Part 1). To bring this up within the Yocto environment (as opposed to just doing a “make”), you have to use the command:
bitbake -c menuconfig virtual/kernel
that puts you into the following screen:
Then you have to click on “Kernel hacking” and then turn on “Kernel debugging”, and also click on “Compile-time checks and compiler options” and turn on “Compile the kernel with debug info”:
Then, save the config, and run bitbake core-image-minimal as is normal.
Eureka! The vmlinux file is now 218MB in size, compared to only 25MB earlier:
Then, when I loaded it in via SourcePoint’s Load Program dialog box, and also pointed the “Source Path…” button to the kernel-source folder as the Debug Path (I had copied the kernel-source folder to my PC, originally from VirtualBox’s Ubuntu poky/build/tmp/work-shared/intel-corei7-64 directory), eureka, I got the source code:
You can see in the top left Code window, a mix of assembly language and source code (right at the beginning of the rtl8169_hw_reset function) as well as all related symbols. Not bad, huh?
Do you want to learn more about the MinnowBoard and experience a fascinating journey through UEFI, x86 architecture, Linux, and other topics? Download the first 31 chapters of our eBook here: The MinnowBoard Chronicles.
