Discussion:
debug of multithreaded program turns to be impossible during execution
(too old to reply)
Avi Gozlan
2014-07-31 09:27:27 UTC
Permalink
Hello,

I'm debugging a multithreaded program using GDB. Various threads are created and exit during execution. GDB copes successfully with the multiple threads (e.g. 'thread apply 3 bt' reports successfully backtrace of thread 3), but then at one point loses control of the program:

(gdb) c
Continuing.
[New Thread 0x41401940 (LWP 13370)]
[Thread 0x40a00940 (LWP 13333) exited]
[New Thread 0x41e02940 (LWP 13371)]
...
[New Thread 0x45a08940 (LWP 13377)]
[New Thread 0x46409940 (LWP 13378)]
[Thread 0x41e02940 (LWP 13371) exited]
[Switching to Thread 0x45a08940 (LWP 13377)]
Cannot remove breakpoints because program is no longer writable.
Further execution is probably impossible.
0x00002aaaab98348f in start_thread () from /lib64/libpthread.so.0
ptrace: No such process.

Surprisingly, although debugging is no more possible, the program is not dead:

(gdb) q
A debugging session is active.

Inferior 1 [process 13296] will be killed.

Quit anyway? (y or n) y
[Thread 0x46409940 (LWP 13378) exited]
[Thread 0x45a08940 (LWP 13377) exited]
...
[Thread 0x2aaaaccbe580 (LWP 13296) exited]

Some details:
GDB version: 7.6
OS: RHEL 5.1 based
GLIBC: glibc-2.5-18
/lib64/libthread-1.0.so is not stripped.

Your assistance will be appreciated.
Thanks,
Avi
Pedro Alves
2014-07-31 10:17:25 UTC
Permalink
Post by Avi Gozlan
Hello,
(gdb) c
Continuing.
[New Thread 0x41401940 (LWP 13370)]
[Thread 0x40a00940 (LWP 13333) exited]
[New Thread 0x41e02940 (LWP 13371)]
...
[New Thread 0x45a08940 (LWP 13377)]
[New Thread 0x46409940 (LWP 13378)]
[Thread 0x41e02940 (LWP 13371) exited]
[Switching to Thread 0x45a08940 (LWP 13377)]
Cannot remove breakpoints because program is no longer writable.
Further execution is probably impossible.
0x00002aaaab98348f in start_thread () from /lib64/libpthread.so.0
ptrace: No such process.
I have a vague recollection that this was fixed a while ago. Like,
GDB was picking the wrong thread to access memory, and it ended up
picking a running or exited thread, and that fails (in either
case ptrace complains the same say).
Post by Avi Gozlan
GDB version: 7.6
Is there a chance you could try a more recent GDB version? Ideally
7.8, which we just released.

Thanks,
Pedro Alves
Avi Gozlan
2014-08-14 15:52:40 UTC
Permalink
GDB 7.8 yields a similar behavior:

(gdb) c
Continuing.
...
[New Thread 0x43c05940 (LWP 3311)]
[New Thread 0x44606940 (LWP 3312)]
...
[Thread 0x44606940 (LWP 3312) exited]
[Switching to Thread 0x43c05940 (LWP 3311)]
Cannot remove breakpoints because program is no longer writable.
Further execution is probably impossible.
0x00002aaaab98348f in start_thread () from /lib64/libpthread.so.0
ptrace: No such process.
(gdb) [Thread 0x46409940 (LWP 3315) exited]
[Thread 0x45a08940 (LWP 3314) exited]
[Thread 0x43c05940 (LWP 3311) exited]
[Inferior 1 (process 3268) exited normally]

The only difference is that in 7.8 case, GDB continues automatically after giving the ptrace message.

Avi

-----Original Message-----
From: Pedro Alves [mailto:***@redhat.com]
Sent: Thursday, July 31, 2014 1:17 PM
To: Avi Gozlan; ***@sourceware.org
Subject: Re: debug of multithreaded program turns to be impossible during execution
Post by Avi Gozlan
Hello,
(gdb) c
Continuing.
[New Thread 0x41401940 (LWP 13370)]
[Thread 0x40a00940 (LWP 13333) exited] [New Thread 0x41e02940 (LWP
13371)] ...
[New Thread 0x45a08940 (LWP 13377)]
[New Thread 0x46409940 (LWP 13378)]
[Thread 0x41e02940 (LWP 13371) exited] [Switching to Thread
0x45a08940 (LWP 13377)] Cannot remove breakpoints because program is
no longer writable.
Further execution is probably impossible.
0x00002aaaab98348f in start_thread () from /lib64/libpthread.so.0
ptrace: No such process.
I have a vague recollection that this was fixed a while ago. Like, GDB was picking the wrong thread to access memory, and it ended up picking a running or exited thread, and that fails (in either case ptrace complains the same say).
Post by Avi Gozlan
GDB version: 7.6
Is there a chance you could try a more recent GDB version? Ideally 7.8, which we just released.

Thanks,
Pedro Alves


Email secured by Check Point.
Pedro Alves
2014-08-19 16:05:51 UTC
Permalink
Post by Avi Gozlan
(gdb) c
Continuing.
...
[New Thread 0x43c05940 (LWP 3311)]
[New Thread 0x44606940 (LWP 3312)]
...
[Thread 0x44606940 (LWP 3312) exited]
[Switching to Thread 0x43c05940 (LWP 3311)]
Cannot remove breakpoints because program is no longer writable.
Further execution is probably impossible.
0x00002aaaab98348f in start_thread () from /lib64/libpthread.so.0
ptrace: No such process.
(gdb) [Thread 0x46409940 (LWP 3315) exited]
[Thread 0x45a08940 (LWP 3314) exited]
[Thread 0x43c05940 (LWP 3311) exited]
[Inferior 1 (process 3268) exited normally]
The only difference is that in 7.8 case, GDB continues automatically after giving the ptrace message.
Any chance you could try your reproducer with a more recent
system? Mainly, I'm wondering whether kernel/glibc are at play here.

Also, a run with:

set debug infrun 1
set debug target 1
set debug lin-lwp 1

may help diagnose the issue.

Thanks,
Pedro Alves

Loading...