"*** stack smashing detected ***"

This sounds like a genuine internal error in adv_jtag_bridge. The first thing I would do in seeing an error like that is to make clean; make, then try again - it may be that some of the object files were built with different options, causing the crash. Try this first.

But, this may be an actual software bug in adv_jtag_bridge. I'm not sure why the JSP module is doing a transaction at that point at all - it should only poll when the CPU is running, and if you just set $pc=0x100, then the CPU should not be running. And even if it is somehow working the way it's supposed to, it should obviously not be stomping on its own heap.

The backtrace still does not have line numbers, which would be a big help in finding the problem. I would also be curious to see what parameters got passed into the final jsp_transact call that crashed. Can you send the core file and your adv_jtag_bridge executable to my opencores email address? I may be able to work it out from there.

Nathan, Sending e-mails to your opencores account results in delivery failure ... Here's my previous email with the files requested ... Delivery to the following recipient failed permanently: nathan.yawn@opencores.org Technical details of permanent failure: Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 550 550 5.1.1 : Recipient address rejected: User unknown in local recipient table (state 13). ----- Original message ----- Nathan, I did a clean compile for both rtl and adv_jtag app with the same options. However, I'm face with the same issues ... I'm attaching the RTL and s/w files as requested. Furthermore, there are 2 points that I'd like to mention: [1] Cosmin Gorgovan (lgeek) did a re-run of his altera de2-115 port with the latest versions of the RTLs and has faced the same issues. [2] My memory device is of the size 64 MB which gives me 16,777,216 x 32-bit addresses => a memory range from 0x0000_0000 to 0x0100_0000 ... however, after loading linux GDB reports that start address = 0xc000_0000 ... isn't this out of my memory range ??? and besides the base address of 0xc0 is not assigned to any peripheral in orpsoc ... Am I missing something here ??? [3] I get some warnings when compiling adv_debug_sys ...is this expected ??? I've attached the compililation log with this e-mail. Many thanks for your help. Regards, AR

Hi Nathan,

Any updates on this?

Regards,
AR

Sorry, been busy with my day job.

First, there is no core file I could find in the files you uploaded, I would need that to examine the crash fully.

Second, I have examined the adv_debug_sys code, and have found a case where this sort of crash is possible. It does not explain why it was trying to do a JSP transaction when the CPU wasn't running, but I have a patch that may allow the program to run. Save everything after this paragraph to a patch file, and apply it in the adv_jtag_bridge directory. The rebuild and try again.


===================================================================
RCS file: /Win7/cvsroot/Bennu/Software/OpenRisc_Devel/adv_jtag_bridge/adv_dbg_commands.c,v
retrieving revision 1.11
diff -B -u -r1.11 adv_dbg_commands.c
--- adv_dbg_commands.c 10 Oct 2010 21:36:58 -0000 1.11
+++ adv_dbg_commands.c 10 May 2012 03:00:32 -0000
@@ -779,7 +779,9 @@
char indata[10];
unsigned char stopbit = 0, startbit = 0, wrapbit;
int bytes_free;
+#ifdef ENABLE_JSP_MULTI
int i;
+#endif

if(*bytes_to_send > 8)
xmitsize = 8;
@@ -849,6 +851,14 @@
debug("jsp got remote sizes 0x%X\n", indata[0]);

*bytes_received = (indata[0] >> 4) & 0xF; // bytes available is in the upper nibble
+
+ // We may get bad data, with bytes_received > 8. Clamp this.
+ if(*bytes_received > 8)
+ {
+ fprintf(stderr, " JSP warning: got bad # of bytes available: %i\n", *bytes_received);
+ *bytes_received = 8;
+ }
+
memcpy(data_received, &indata[1], *bytes_received);

bytes_free = indata[0] & 0x0F;
@@ -877,6 +887,13 @@
if(bytes_free if((*bytes_received) > xmitsize) xmitsize = *bytes_received;

+ // We may get bad data, with bytes_received > 8. Clamp this.
+ if(*bytes_received > 8)
+ {
+ fprintf(stderr, " JSP warning: got bad # of bytes available: %i\n", *bytes_received);
+ *bytes_received = 8;
+ }
+
memset(outdata, 0, 10);
memcpy(outdata, data_to_send, xmitsize); // use larger array in case we need to send stopbit

hi Nathan,

[1] Tried applying your patch using 'git apply 0000.patch' ... it results in 'corrupt data at line 7' ... looked at your patch and applied the codes manually ... and did a clean compile ...

[2] same errors still persist ... jtag crashes with the same error msgs ... no luck there!

[3] I'm not being able to generate a core file for some unknown reason ... i've tried every trick in the book (ulimit -c unlimited etc) ... any ideas?

[4] I've noticed something strange ... have a look!

(gdb) x 0x100 // fine
0x100: 0x1880c014

(gdb) x 0xc0000000 // as expected ... 0xc isn't assigned to any IP core
0xc0000000 : Cannot access memory at address 0xc0000000

(gdb) x 0x100 // ???
0x100: Cannot access memory at address 0x100

is this normal ???

regards,
AR

[1-3] Blargh. I'd really like to see what line of the program is supposedly overwriting the stack. Core file, running adv_jtag_bridge in GDB, running it in valgrind with the memcheck tool...anything to find that line would help.

It might be a minor help if you enabled debug logging in that file. At the top of adv_dbg_commands.c, there is a #define debug() that has half the line commented out. Remove the "//", this will enable debug logging in that file (there will be a lot of it). Rebuild, run it again, and maybe something useful will appear at the console before it crashes.

[4] Depending on your SoC implementation, this might actually be expected behavior. If I remember correctly, the WishBone bus requires some sort of response to every transaction. The simple WishBone implementations (conbus, conmax, traffic cop) all require a valid peripheral at every addressed accessed to provide that response. So, if you try to access a WishBone address that has nothing there, the bus may hang forever waiting for a response. I don't know if the bus used by ORPSoC has this behavior or not. But, that would explain why you got no response from a valid address after accessing an invalid address.

Nathan,

see if this helps ... you were right - with the debug option enabled, a lot of debug info are printed ... The msgs below were printed just before the app crashed ... it may be of some interest ...

CRC OK!
ctrl wr idx 0 dat 0x0
select module 3
jsp doing 9 bytes, xmitsize 0
jsp got remote sizes 0x8
*** stack smashing detected ***: /home/openrisc/adv_jtag_bridge/lgeek-adv_debug_sys-patched/Software/adv_jtag_bridge/adv_jtag_bridge terminated

Backtrace report:

(gdb) bt
#0 0x00130416 in __kernel_vsyscall ()
#1 0x00188c8f in raise () from /lib/i386-linux-gnu/libc.so.6
#2 0x0018c2b5 in abort () from /lib/i386-linux-gnu/libc.so.6
#3 0x001bedfc in ?? () from /lib/i386-linux-gnu/libc.so.6
#4 0x002428d5 in __fortify_fail () from /lib/i386-linux-gnu/libc.so.6
#5 0x00242887 in __stack_chk_fail () from /lib/i386-linux-gnu/libc.so.6
#6 0x08054094 in adbg_jsp_transact (bytes_to_send=0xb77eb1f4,
data_to_send=0xb77eb1fc "944\n\n\r", bytes_received=0xb77eb1f8,
data_received=0xb77eb204 "") at adv_dbg_commands.c:925
#7 0x08054bb6 in dbg_serial_sndrcv (bytes_to_send=0xb77eb1f4,
data_to_send=0xb77eb1fc "944\n\n\r", bytes_received=0xb77eb1f8,
data_received=0xb77eb204 "") at dbg_api.c:681
#8 0x0805ae7d in jsp_hardware_transact () at jsp_server.c:527
#9 0x0805b47d in jsp_server (arg=0x0) at jsp_server.c:351
#10 0x00137d31 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#11 0x0022d0ce in clone () from /lib/i386-linux-gnu/libc.so.6

Don't know if this helps but that's the best i could come up with ... still trying to generate a core file but no luck yet ...

Regards,
AR

Ah, there it is:

#6 0x08054094 in adbg_jsp_transact (bytes_to_send=0xb77eb1f4,
data_to_send=0xb77eb1fc "944\n\n\r", bytes_received=0xb77eb1f8,
data_received=0xb77eb204 "") at adv_dbg_commands.c:925

Could you please confirm that line 925 in your version of that file just has the close-brace of the adbg_jsp_transact() function?

That makes some sense, actually...there's nothing doing run-time stack checking on every memory access, but the function epilogue code checks it before it uses the data on the stack to return to the caller. Unfortunately, that doesn't tell us where the stack smashing is happening. But, I have a suspicion. If you can, try two more things:

1. In cable_ft245.c, on line 45, there should be an "#if 1". Change it to "#if 0". Rebuild, and try again. It will be terribly slow, but if it works, that will point a finger at a possible location for the error.

2. Undo #1, then try to run adv_jtag_bridge with Valgrind (using the memcheck tool). I believe this will do run-time checking on every memory access, and may be able to flag exactly where the stack smashing is happening. If you get Valgrind to log this error, send me the logs.

This has at least given me a new idea of what bit of code the error may be in, I'll have a look and see if anything jumps out at me.

I may have found the cause of the stack smashing. A proper fix will have to wait until I have some more time, but as a temporary workaround:

In adv_dbg_commands.c, lines 778 and 779, change the sizes of the outdata and indata arrays to 16.

Let me know if that works out.

I also encounter the problem:
Cannot access memory at address 0xc0000000
(gdb) spr npc 0x100
SYS.NPC (SPR0_16) set to 256 (0x100), was: 1540 (0x604)
(gdb) set $pc = 0x100
Cannot access memory at address 0xc0000000



I have try changed the outdata and indata to 16, bu the situations remains,
Is there temporary workaround way?