Skip to content

Debugging Linux Software

Anton Kolesov edited this page Apr 20, 2017 · 15 revisions

Please see our documentation site for the latest version of this page.

This article describes how to debug user-space applications on the Linux on ARC.

Building toolchain

In most cases it should be enough to use binary distribution of GNU Toolchain for ARC, which can be downloaded from https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases. If toolchain in binary distribution doesn't fit some particular requirements, then instruction to build toolchain from source can be found in README.md file in the toolchain repository.

Building Linux

The simple guide to build kernel can be found in this wiki: . More instructions can be found in ARC Linux wiki and in the Internet in general.

Configuring target system

Information in this section is not specific to ARC, it is given here just for convenience - there are other ways to achieve same result.

Configuring networking

Ethernet model is not available in standalone nSIM simulation.

By default target system will not bring up networking device. To do this:

[arclinux] # ifconfig eth0 up

If network to which board or virtual platform is attached has a DHCP server, then run DHCP client:

[arclinux] # udhcpc

If there is no DHCP server, then configure networking manually:

[arclinux] # ifconfig eth0 <IP_ADDRESS> netmask <IP_NETMASK>
[arclinux] # route add default gw <NETWORK_GATEWAY> eth0

Where <IP_ADDRESS> is an IP address to assign to ARC Linux, <IP_NETMASK> is a mask of this network, <NETWORK_GATEWAY> is default gateway of network.

To gain access to the Internet DNS must servers must be configured. This is usually not required when using DHCP, because in this case information about DNS servers is provided via DHCP. To configure DNS manually, create /etc/resolv.conf which lists DNS servers by IP. For example:

nameserver 8.8.8.8
nameserver 8.8.4.4

That will connect ARC Linux to the network.

Configuring NFS

To ease process of delivering target application into the ARC Linux it is recommended to configure NFS share and mount it on the ARC Linux. Refer to this article for details on how to bring up NFS share on the Ubuntu host. If you already has a working NFS share connect to it with following command:

[arclinux] # mount -t nfs -o nolock,rw <NFS_SERVER_IP>:<NFS_SHARE_PATH> /mnt

Network share will be mounted at /mnt.

Additional services

Another thing that might be useful is to have network services like telnet, ftp, etc, that will run on ARC Linux. First make sure that desired service is available in the Busybox configuration. Run make menuconfig from Busybox directory or make busybox-menuconfig if you are using Buildroot. Make sure that "inetd" server is enabled. Select required packages (telnet, ftpd, etc) and save configuration. Rebuild busybox (run make busybox-rebuild if you are using Buildroot).

Then configure inetd daemon. Refer to inetd documentation to learn how to do this. In the simple case it is required to create /etc/inetd.conf file on the target system with following contents:

ftp     stream  tcp nowait  root    /usr/sbin/ftpd      ftpd -w /
telnet  stream  tcp nowait  root    /usr/sbin/telnetd   telnetd -i -l /bin/sh

Thus inetd will allow connections to ftpd and telnetd servers on the target system. Other services can be added if required.

Rebuild and update rootfs and vmlinux. Start rebuilt system and run inetd to start inetd daemon on target:

[arclinux] # inetd

Debugging applications with gdbserver

It is assumed that one or another way application to debug is on to the target system. Run application on target with gdbserver:

[arclinux] # gdbserver :49101 <application-to-debug> [application arguments]

TCP port number could any port not occupied by another application. Then run GDB on the host:

$ arc-linux-gdb <application-to-debug>

Then set sysroot directory path. Sysroot is a "mirror" of the target system file system: it contains copies of the applications and shared libraries installed on the target system. Path to the sysroot directory should be set to allow GDB to step into shared libraries functions. Note that shared libraries and applications on the target system can be stripped from the debug symbols to preserve disk space, while files in the sysroot shouldn't be stripped. In case of Buildroot-generated rootfs sysroot directory can be found under <BUILDROOT_OUTPUT>/staging.

(gdb) set sysroot <SYSROOT_PATH>

Then connect to the remote gdbserver:

(gdb) target remote <TARGET_IP>:49101

You can find <TARGET_IP> via running ifconfig on the target system. TCP port must much the one used when starting up gdbserver. It is important that sysroot should be set before connecting to remote target, otherwise GDB might have issues with stepping into shared libraries functions.

Then you can your debug session as usual. In the simplest case:

(gdb) continue

Note that there is a known limitation of gdbserver - it is not safe to debug multiprocess application with it. Problem is that when child if forked, it still shares code pages with parent, therefore software breakpoints set in the parent process might be hit by the child process should it execute the same code path. In this case child process will crash due to unexpected breakpoint. This is a generic problem with gdbserver, that is not specific to ARC port of GDB - it can be reproduced with gdb/gdbserver for x86_64.

Debugging applications with native GDB

Starting from GNU Toolchain for ARC release 2014.08 it is possible to build full GDB to run natively on ARC Linux. Starting from GNU Tooolchain for ARC release 2015.06 native GDB is automatically built for uClibc toolchain (can be disabled by --no-native-gdb option). In GNU Toolchain prebuilt tarballs native GDB binary can be found in sysroot directory: arc-snps-linux-uclibc/sysroot/usr/bin/gdb

With native GDB it is possible to debug applications the same way as it is done on the host system without gdbserver.

When choosing between gdbserver and native GDB, following pros and cons should be considered.

Pros of native GDB:

  • Overhead for network communication between GDB and gdbserver is removed, theoretically improving debugging performance.
  • Some features might be not implemented in the gdbserver.
  • As described in gdbserver section - gdbserver cannot be safely used to debug applications that use fork(). Therefore native GDB is the debugger of choice for multiprocess applications.
  • There is no need for a second host to perform debugging session, since everything is on the target system.

Cons:

  • It is required that applications on target system should have debugging symbols (unless you are so hardcore that you don't need them). Debugging symbols, especially in the most verbose case occupy significant disk space. Depending on the type of target hardware this might be or might not be a thing to consider. Usually this can be ignored in case of virtual prototypes, and is hardly a problem with development systems, however disk space is probably very limited on the production systems. Large rootfs size also means increased time required to load rootfs into the target memory.
  • Not only debugging symbols will take noticeable disk space, but also GDB will also read them intensively, so if target file system has a low performance, this might be noticeable.
  • Full GDB on target requires more computational power than gdbserver. This might offset all of the gains from exclusion of the networking layer.

In general it is highly dependent on target system properties and developer needs whether gdbserver or native GDB is better and it is up to the software developer to decide what is better in their particular case.

Clone this wiki locally