Hi, I have successfully built the or32-elf toolchain, or1ksim, and the linux-2.6.24 kernel. I have verified that the or1ksim architectural simulator is working with my linux kernel by doing:
or32-elf-sim -f or1ksim_linux.cfg vmlinux
I have also been able to verify that my RTL compiles and simulates properly on ModelSim by doing:
$ cd sim/run
$ make rtl-tests SIMULATOR=vsim

What I would like to do next is boot the linux kernel in RTL simulation. To give it a go, I tried creating the flash and sram images below, and firing up the simulation.
$ ../../sw/utils/bin2hex vmlinux.bin 1 -size_word > flash.in
$ ../../sw/utils/bin2vmem vmlinux.bin > sram.vmem

This did not work for me. I assume that my code is is not correct at 0x100?
If anyone could steer me in the right direction, I would appreciate it!

Thanks!
Jason

It's working for me. I don't know how far it goes into booting Linux (it takes a long time for it to full boot, many many hours) but I can see in the execution log that it's configured the MMU and performing intialisation of some kind.

Check the configuration of the compiled Linux kernel. I believe by default the "Boot from FLASH" option is enabled, as well as a frequency for the simulator of 100MHz, while ORPSoC is at 25MHz.

In the kernel's menuconfig screen, under "Processor type and features", uncheck "Boot from FLASH". You can also set the expected clock rate in the same screen (change it from 100 to 25MHz under "System clock frequency [MHz]".) Everything else should be OK, except for perhaps if data cache is enabled (I think it's disabled in ORPSoC by default.) Recompile and try re-running like that.

You could, if you wanted, actually perform the boot from FLASH in the simulator, but I would advise against it (and is the whole reason ORPSoC defaults to internal RAM magically preloaded with the application) because it would take a while to load the 3.6MBytes of kernel from FLASH into the SDRAM and execute. This is what the USE_SDRAM option in ORPSoC enables. The "Boot from FLASH" option in the kernel enables some code, being executed from FLASH, to copy itself into some memory elsewhere and then jump resume execution from there. Hence it breaks when it thinks it's in FLASH and wants to copy itself elsewhere. I'd stick to the internal SRAM, and in that case you only need to provide the sram.vmem file for ORPSoC (don't worry about the flash.in file).

Hi Julius,

Thanks for the quick response. I recompiled the linux kernel to disable boot from flash, and changed the clock frequency to 25 MHz. I recompiled the RTL with +define+USE_SDRAM, and I'm re-running my RTL simulation now.

Just to make sure I have the correct expectation, I'm expecting to see console messages from booting linux (like we see using or1ksim) coming from the UART. Is this what you see in your working RTL simulation?

Regards,
Jason

A couple questions:

1. When I ran all sim-tests and rtl-tests, the sim tests are run using the or1ksim-orpsocv2.cfg file. This file differs significantly from the or1ksim_linux.cfg file that I had used to simulate booting linux. Since my rtl-tests matched the sim-tests results, I am wondering if I need to build a linux kernel that works with the or1ksim-orpsocv2.cfg file? Any info on this would be much appreciated.

2. Is there any documentation on how to sync up RTL defines with or1ksim cfg settings?

Thanks!
Jason

There's two ways the system can boot in ORPSoCv2. One is from the large "internal" SRAM that gets preloaded with the sram.vmem file (default) or from FLASH (which is preloaded with the program), which copies the program into SDRAM and then executes from it. There's no way to immediately preload the SDRAM, which is probably what you want to do as simulating the copying of the app via SPI from FLASH isn't the most interesting thing. So I would recommend just using the internal SRAM memory as your memory model, to speed up simulation (although it would skew your performance specs, if that's what your interested in, because the internal SRAM takes less cycles to access)


Yes you should see the UART printout if everything is working correctly. In order for the UART decoder to be included in the simulation you need to define UART_PRINTF for the testbench.


There does appear to be several differences between those files, but mainly I think it's just the clock rate and locations of memory.

The config file included with Linux sets up a FLASH memory at 0xf0000000, whereas in ORPSoC there isn't a directly addressable FLASH memory server, (it's used via an SPI controller, and is a little more involved than just requesting an address to read from) so the config file reflects this, although there does appear to be a FLASH memory section defined from 0x04000000, but that isn't used by the software. None of the other peripherals really matter (most are not implemented in ORPSoCv2).

So, I think to make a Linux image that boots in ORPSoCv2's RTL model you just need to change the frequency, and disable the boot from FLASH option. The RTL and testbench should be configured so it's loaded into the internal SRAM model, and then enable UART output. Potentially the kernel tries to initialise the ethernet MAC core at some stage, so maybe you want to run the model with that enabled too.

One of the things I did get running a while back was Linux in the cycle-accurate model. I remember I got that to boot, so unless you have a fast simulator, I'd recommend using that by just doing a 'make prepare-vlt' (or 'make prepare-vlt-profiled' if you have a few extra minutes) and then you can even just load the vmlinux elf directly into the simulator that way I think.

There's two ways the system can boot in ORPSoCv2. One is from the large "internal" SRAM that gets preloaded with the sram.vmem file (default) or from FLASH (which is preloaded with the program), which copies the program into SDRAM and then executes from it. There's no way to immediately preload the SDRAM, which is probably what you want to do as simulating the copying of the app via SPI from FLASH isn't the most interesting thing. So I would recommend just using the internal SRAM memory as your memory model, to speed up simulation (although it would skew your performance specs, if that's what your interested in, because the internal SRAM takes less cycles to access)


Yes you should see the UART printout if everything is working correctly. In order for the UART decoder to be included in the simulation you need to define UART_PRINTF for the testbench.


There does appear to be several differences between those files, but mainly I think it's just the clock rate and locations of memory.

The config file included with Linux sets up a FLASH memory at 0xf0000000, whereas in ORPSoC there isn't a directly addressable FLASH memory server, (it's used via an SPI controller, and is a little more involved than just requesting an address to read from) so the config file reflects this, although there does appear to be a FLASH memory section defined from 0x04000000, but that isn't used by the software. None of the other peripherals really matter (most are not implemented in ORPSoCv2).

So, I think to make a Linux image that boots in ORPSoCv2's RTL model you just need to change the frequency, and disable the boot from FLASH option. The RTL and testbench should be configured so it's loaded into the internal SRAM model, and then enable UART output. Potentially the kernel tries to initialise the ethernet MAC core at some stage, so maybe you want to run the model with that enabled too.

One of the things I did get running a while back was Linux in the cycle-accurate model. I remember I got that to boot, so unless you have a fast simulator, I'd recommend using that by just doing a 'make prepare-vlt' (or 'make prepare-vlt-profiled' if you have a few extra minutes) and then you can even just load the vmlinux elf directly into the simulator that way I think.

Thanks for the info, Julius. I tried the cycle-accurate simulation, but still can't boot this kernel. See below:

$ Vorpsoc_top -f vmlinux -c -e 5000000000

             SystemC 2.2.0 --- Mar  8 2010 12:24:57
        Copyright (c) 1996-2006 by all Contributors
                    ALL RIGHTS RESERVED
* Crash monitor enabled
* Loading program from /home/jrothfus/proj7/OpenRISC.1/MOF/or32-build/b-linux/linux-2.6.24/vmlinux
* Section: .text,* vaddr: 0xc0000000,* paddr: 0x0* offset: 0x00002000,* size: 0x0021e2dc
* Section: .text.init.refok,* vaddr: 0xc021e2dc,* paddr: 0x21e2dc* offset: 0x002202dc,* size: 0x00000410
* Section: .exit.text.refok,* vaddr: 0xc021e6ec,* paddr: 0x21e6ec* offset: 0x002206ec,* size: 0x00000214
* Section: .rodata,* vaddr: 0xc021f000,* paddr: 0x21f000* offset: 0x00221000,* size: 0x0002cd3c
* Section: __param,* vaddr: 0xc024bd3c,* paddr: 0x24bd3c* offset: 0x0024dd3c,* size: 0x00000280
* Section: __ex_table,* vaddr: 0xc024c000,* paddr: 0x24c000* offset: 0x0024e000,* size: 0x00000c60
* Section: .data,* vaddr: 0xc024e000,* paddr: 0x24e000* offset: 0x00250000,* size: 0x00019328
* Section: .data.cacheline_aligned,* vaddr: 0xc0267340,* paddr: 0x267340* offset: 0x00269340,* size: 0x000000e0
* Section: .data.init_task,* vaddr: 0xc0268000,* paddr: 0x268000* offset: 0x0026a000,* size: 0x00002000
* Section: .init.text,* vaddr: 0xc026a000,* paddr: 0x26a000* offset: 0x0026c000,* size: 0x00017194
* Section: .init.data,* vaddr: 0xc0281194,* paddr: 0x281194* offset: 0x00283194,* size: 0x00000898
* Section: .init.setup,* vaddr: 0xc0281a30,* paddr: 0x281a30* offset: 0x00283a30,* size: 0x00000234
* Section: .initcall.init,* vaddr: 0xc0281c64,* paddr: 0x281c64* offset: 0x00283c64,* size: 0x00000194
* Section: .con_initcall.init,* vaddr: 0xc0281df8,* paddr: 0x281df8* offset: 0x00283df8,* size: 0x00000008
* Section: .init.ramfs,* vaddr: 0xc0282000,* paddr: 0x282000* offset: 0x00284000,* size: 0x00002000
* Section: .initrd,* vaddr: 0xc0284000,* paddr: 0x284000* offset: 0x00286000,* size: 0x00200000
SystemC: simulation stopped by user.
* Or1200Monitor: simulated 5 s, time elapsed: 931.61 seconds
* Or1200Monitor: simulated 125000000 clock cycles, executed at approx 134kHz
* Or1200Monitor: simulated 29761897 instructions, insn/sec. = 31946.7

However, the uart-nocache.or32 test does pass and match rtl-simulation. Also, 18/18 vlt-tests pass.

I am guessing that this means that my vmlinux is bad? If possible, can someone post a working .config to allow booting linux in rtl or sysc simulation? Any advice on debugging this problem is welcome.

Thanks!
Jason

The config I'm using should be the same as this the one on the Linux and BusyBox guide: http://opencores.org/openrisc,linux_busybox

I've compiled that and are now running it in the simulation, but have spotted an error:

The problem is to do with a simple thing very early on (in the first few functions it calls) it tries to convert the address of the little "Ok, booting" string from a virtual to physical address, passing it through a function called tophys() which subtracts the Linux kernel's virtual address base (0xc0000000), however this address is already a physical address, so we end up with an invalid address, which gets put out on the bus, which of course doesn't know what to do and errors, or something like that, so it then goes to 0x200, a bus error.

If I disable that tophys() there (just copy the value instead) I then see a couple more calls which trip up the boot process. Not sure if'll you'll see these problems, but I encountered them here, so I thought it'd note it.

I had to disable a few tophys() calls in both the reset vector and the _start function in arch/or32/kernel/head.S, and it looked like it booted and ran for a while.

I launched the cycle accurate model like so:



And then monitored where it was by doing this in another open console:



This isn't pretty, but it's quick and easy, you'll get this output:



(I stripped the little address and ascii decoding from either side of the data to avoid confusion.)

The line you want to look at is the middle one (at least this seemed to be where tail got its data from each time). Essentially when you do a binary log in the cycle accurate model, with no register data, it dumps out 8 bytes of instruction counter, then 4 byte PC and 4 byte instruction, and 1 byte indicating whether it's an exception or not, so 25 bytes in all.

The bytes are printed LSB first, so they're backwards as you read them from hexdump.

But here, reading that middle line we can tell we're at instruction number 12,906,031 (or 0xc4ee2f), the PC is 0x001355cc, and the instruction is 0xb8840001 (verified by doing a or32-elf-objdump -d vmlinux | grep 1355cc).

The reason i monitor the simulation this way is because when the sim is running and creating a human-readable log (without the -b option) it's slower. Also, if you disable the dumping of the GPR values that speeds it up too. Of course you can always connect GDB via RSP to control it that way too, but here I was just trying to see if the thing was working. And it looks like it it's doing something other than crashing.

Although, if I ran it for a while I noticed it went to the kernel panic function, which I guess is not a good thing. But, I guess to debug that you would connect GDB and run it and set a break there, and see how it ended up going there.

An aside: to convert those binary logs back to humanreadable form there's a tool I wrote in sw/utils called binlog2readable - it lets dump all or part of the binary log to see what it was doing.

I'm getting different results. The sim starts up at the reset vector 0x100, jumps to 0x223c, runs a few instructions, then a bus error occurs.

One thing I noticed is that the _tng_start_kernel symbol (which I believe is supposed to be the reset vector) is at address 0x00009c00. This doesn't seem right to me. If I look at the uart-nocache.or32 symbols, I see that the reset vector is mapped to 0x100 and all other exceptions are mapped properly. Is there something wrong with the linker script for the linux kernel?

Thanks,
Jason

I see now that the reset vector is in arch/or32/kernel/head.S. I mistakenly thought it was in arch/or32/kernel/entry.S.

The kernel starts booting (then fails with a bus error) in the SystemC model when I unset the CONFIG_OR32_MC_INIT parameter in the .config file.

I saw this error before.

For me it was caused by those tophys() functions being called on an address that wasn't in the virtual address space. I removed the calls to tophys() which were being passed linker-defined constants. Here are a couple of examples of what I did in head.S:


/*
* set up initial ksp and current
*/
LOAD_SYMBOL_2_GPR(r1,_init_thread_union+0x2000) // setup kernel stack
LOAD_SYMBOL_2_GPR(r10,_init_thread_union) // setup current
/* tophys (r31,r10) -- jb*/
l.and r31, r10, r10
l.sw TI_KSP(r31), r1




clear_bss:
LOAD_SYMBOL_2_GPR(r24, ___bss_start)
LOAD_SYMBOL_2_GPR(r26, __end)
/*tophys(r28,r24)
tophys(r30,r26) --jb*/
l.and r28, r24, r24
l.and r30, r26, r26




LOAD_SYMBOL_2_GPR(r4,_string_ic_enable)
/*tophys (r3,r4) --jb*/
and r3, r4, r4



In fact, those were the 3 places where it was getting caught while booting, perhaps there's more later. I don't know the elegant way of fixing it - I think most of the things it's converting from the 0xc0000000 address space are values which the linker script ultimately controls. You can try just making tophys() not do the actual subtraction of 0xc0000000 from the value (function is in include/or32-asm/or32_asm.h) but that may break things.

I wiped out my kernel build with the tophys() changes and started from scratch. I untarred the linux kernel, applied the or32 patch. I then did "make menuconfig" and disabled "Boot From Flash", "Initialize Memory Controller", and changed the CPU clock speed to 25MHz. For whatever reason, now things are *much* better. The SystemC model is now matching the RTL model. I am able to boot up to the following place, but then get the same error (shown below) in RTL and SystemC.

Any ideas where to look to fix this issue? I feel like I'm really close now...

Linux version 2.6.24-or32 (jrothfus@mach64) (gcc version 4.2.2) #5 Mon Apr 12 12:11:06 PDT 2010
Detecting Processor units:
  Signed 0x391
Setting up paging and PTEs.
write protecting ro sections (0xc0002000 - 0xc01f0000)
Setting up identical mapping (0x80000000 - 0x90000000)
Setting up identical mapping (0x92000000 - 0x92002000)
Setting up identical mapping (0xb8070000 - 0xb8072000)
Setting up identical mapping (0x97000000 - 0x97002000)
Setting up identical mapping (0x99000000 - 0x9a000000)
Setting up identical mapping (0x93000000 - 0x93002000)
Setting up identical mapping (0xa6000000 - 0xa6100000)
Setting up identical mapping (0x1e50000 - 0x1fa0000)
dtlb_miss_handler c00040c8
itlb_miss_handler c00041a8
Early serial console at MMIO 0x90000000 (options '120192')
console [uart0] enabled
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 3953
Kernel command line:
PID hash table entries: 128 (order: 7, 512 bytes)
start_kernel(): bug: interrupts were enabled early
Console: colour dummy device 80x25
Dentry cache hash table entries: 4096 (order: 1, 16384 bytes)
Inode-cache hash table entries: 2048 (order: 0, 8192 bytes)
Memory: 26984k/31744k available (1840k kernel code, 4760k reserved, 241k data, 104k init, 0k highmem)
Mount-cache hash table entries: 1024
Initializing cgroup subsys ns
net_namespace: 64 bytes
NET: Registered protocol family 16
NET: Registered protocol family 2
IP route cache hash table entries: 2048 (order: 0, 8192 bytes)
TCP established hash table entries: 1024 (order: 0, 8192 bytes)
TCP bind hash table entries: 1024 (order: -1, 4096 bytes)
TCP: Hash tables configured (established 1024 bind 1024)
TCP reno registered
checking if image is initramfs...it isn't (bad gzip magic numbers); looks like an initrd
Freeing initrd memory: 2048k freed
eth0: Open Ethernet Core Version 1.0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 2048 (order 0, 8192 bytes)
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
�¿serial8250.0: ttyS0 at MMIO 0x90000000 (irq = 2) is a 16550A
console handover: boot [uart0] -> real [ttyS0]
RAMDISK driver initialized: 16 RAM disks of 8192K size 1024 blocksize
loop: module loaded
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
RAMDISK: ext2 filesystem found at block 0
RAMDISK: Loading 2048KiB [1 disk] into ram disk... done.
BUG: failure at fs/dcache.c:982/d_instantiate()!
Kernel panic - not syncing: BUG!