Subversion Repositories openarty

\documentclass{gqtekspec}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%

%% Filename:    spec.tex

%%

%% Project:     OpenArty, an entirely open SoC based upon the Arty platform

%%

%% Purpose:

%%

%% Creator:     Dan Gisselquist, Ph.D.

%%              Gisselquist Technology, LLC

%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%

%% Copyright (C) 2015-2016, Gisselquist Technology, LLC

%%

%% This program is free software (firmware): you can redistribute it and/or

%% modify it under the terms of  the GNU General Public License as published

%% by the Free Software Foundation, either version 3 of the License, or (at

%% your option) any later version.

%%

%% This program is distributed in the hope that it will be useful, but WITHOUT

%% ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY or

%% FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License

%% for more details.

%%

%% You should have received a copy of the GNU General Public License along

%% with this program.  (It's in the $(ROOT)/doc directory, run make with no

%% target there if the PDF file isn't present.)  If not, see

%% <http://www.gnu.org/licenses/> for a copy.

%%

%% License:     GPL, v3, as defined and found on www.gnu.org,

%%              http://www.gnu.org/licenses/gpl.html

%%

%%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%%

%%

\usepackage{import}

\usepackage{bytefield}

\usepackage{listings}

\project{OpenArty}

\title{Specification}

\author{Dan Gisselquist, Ph.D.}

\email{dgisselq (at)  ieee.org}

\revision{Rev.~0.0}

\begin{document}

\pagestyle{gqtekspecplain}

\titlepage

\begin{license}

Copyright (C) \theyear\today, Gisselquist Technology, LLC

This project is free software (firmware): you can redistribute it and/or

modify it under the terms of  the GNU General Public License as published

by the Free Software Foundation, either version 3 of the License, or (at

your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT

ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY or

FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License

for more details.

You should have received a copy of the GNU General Public License along

with this program.  If not, see \texttt{http://www.gnu.org/licenses/} for a copy.

\end{license}

\begin{revisionhistory}

0.0 &  6/20/2016 & Gisselquist & First Draft \\\hline

0.0 & 10/21/2016 & Gisselquist & More Comments Added\\\hline

0.0 & 11/18/2016 & Gisselquist & Added a getting started section\\\hline

\end{revisionhistory}

% Revision History

% Table of Contents, named Contents

\tableofcontents

\listoffigures

\listoftables

\begin{preface}

\end{preface}

\chapter{Introduction}\label{ch:intro}

\pagenumbering{arabic}

\setcounter{page}{1}

At {\$ 99}, the Arty is a very economical FPGA platform for doing

a lot of things.  It was designed to support the MicroBlaze soft CPU platform,

and as a result it has a lot more memory plus ethernet support.  Put together,

it feels like it was designed for soft--core CPU development.  Indeed, it has

an amazing capability for its price.

Instructions and examples for using the Arty, however, tend to focus on

schematic design development techniques.  While these may seem like an

appropriate way to introduce a beginner to hardware design, these techniques

introduce a whole host of problems.

The first and perhaps biggest problem is that it can be difficult to trouble

shoot what is going on.  This is a combination of two factors.  The first is

that many of the reference schematic designs make use of proprietary IP.  In

an effort to protect both their IP and themselves, companies providing such

IP resources often make them opaque, and difficult to see the internals of.

As a result, it can be difficult to understand why that IP isn't working in

your design.  Further, while many simulation tools exist, only the Xilinx tools

will allow full simulation of Xilinx proprietary IP.  Finally, while it may be

simple to select a part and ``wire'' it up within a schematic, most IP

components have many, many configuration options which are then hidden from the

user within the simplified component.  These options may be the difference

between successfully using the component and an exercise in frustration.

Put together, all of these features of schematic design make the design more

difficult to troubleshoot, and often even impossible to troubleshoot using

open source tools such as Verilator.

Another problem is that schematic based designs often hide their FPGA resource

usage.  They can easily become resource hogs, leaving the designer unaware

of the consequences of what he/she is implementing.  As an example, the memory

interface generated by Xilinx's Memory Interface Generator (MIG) consumes

nearly a full quarter of the Arty's FPGA resources, while delaying responses

to requests by upwards of 250~ns.  Further, while Xilinx touts its MicroBlaze

processor as only using 800--2500 LUTs, the MicroBlaze architecture requires

it be connected to four separate AXI busses, with each of those having

five channels, all with their requests and acknowledgement flags.  These

can therefore easily consume all of the resources within an architecture, before

providing any of the benefit the designer was looking for when they chose to

use an FPGA.

% What is old

%       Arty, XuLA, Learnables using schematic drawing techniques

% What does the old lack?

%       Arty lacks open interfaces, instead using MIG and CoreGen w/ AXI bus

% What is new

%       OpenArty has its own memory interface controller, and runs everything

%       off of an open Wishbone bus structure.

% What does the new have that the old lacks

%

% What performance gain can be expected?

%

Here in this project, we present another alternative.

First, the OpenArty is entirely built in Verilog, and (with the exception of

the MIG controller), it is entirely buit out of OpenSource IP.\footnote{I'm

still hoping to place an open memory controller into this design.  This

controller is written in logic, but does not yet connect to any hardware ports.}

Second, configuration options, such as cache sizes, can be fine tuned via a

CPU options file.

Third, as you will find from examining the RTL sources, this project uses only

one bus, and that bus has ony one channel associated with it: a Wishbone Bus.

This helps to limit the logic associated with trying to read and write from

the CPU, although it may increase problems with fanout.

Finally, because the OpenArty project is made from open source components, the

entire design, together with several of its peripherals, can be simulated using

Verilator.  This makes it possible to run programs on the ZipCPU within the

OpenArty design, and find and examine where such programs (or their peripherals)

fail.

Overall, the goals of this OpenArty project include:

\begin{enumerate}

\item Use entirely open interfaces

        This means not using the Memory Interface Generator (MIG), the

        Xilinx CoreGen IP, etc.

        (This goal has not yet been achieved.)

\item Use all of Arty's on--board hardware: Flash, DDR3-SDRAM, Ethernet, and

        everything else at their full and fastest speed(s).  For example, the

        flash will need to be clocked at 82~MHz, not the 50~MHz I've clocked

        it at in previous projects.  The DDR3 SDRAM memory should also be able

        to support pipelined 32--bit interactions over the Wishbone bus at a

        162~MHz clock.  Finally, the Ethernet controller should be supported

        by a DMA capable interface that can drive the ethernet at its full

        100Mbps rate.

        (Of these, only the ethernet goal has been met.)

\item Run using a 162.5~MHz system clock, if for no other reason than to gain

        the experience of building logic that can run that fast.\footnote{The

        original goal was to run at 200~MHz.  However, the memory controller

        cannot run faster than about 82~MHz.  If we run it at 81.25~MHz and

        double that clock to get our logic clock, that now places us at

        162.5~MHz.  200~MHz is \ldots too fast for DDR3 transfers using the

        Artix--7 chip on the Arty.}

        While the wishbone bus has been upgraded so that it may run at

        200~MHz, the CPU and memory controller cannot handle this speed (yet).

\item Modify the ZipCPU to support an MMU and a data cache, and perhaps even

        a floating point unit.

        (These are still in development.)

\item The default configuration will also include four Pmods: a USBUART,

        a GPS, an SDCard, and an OLEDrgb.

        (These have all been tested, and are known to work.)

\end{enumerate}

I intend to demonstrate this project with a couple programs:

\begin{enumerate}

\item An NTP Server

        While the GPS tracking circuit is in place, and while it appears to be

        able to track a GPS signal to within about 100ns or so, the

        network stack has yet to be built.

\item A ZipOS that can actually load and run programs from the SD Card, rather

        than just a static memory image stored in flash on start-up.

        This will require a functioning memory management unit (MMU), which

        will be a new addition to the ZipCPU created to support this project.

        For those not familiar with MMU's, an MMU translates memory addresses

        from a virtual address space to a physical address space.  This allows

        every program running on the ZipCPU to believe that they own the entire

        memory address space, while allowing the operating system to allocate

        actual physical memory addresses as necessary to support whatever

        program needs more (or less) memory.

        At this point, the MMU has been written and has passed its bench

        testing phase.  It has not (yet) been integrated with the CPU.

\end{enumerate}

\chapter{Getting Started}\label{ch:getting-started}

\section{Building the Core}

%

\section{Building the board support files}

The OpenArty project comes with a series of board support programs that are

designed to run from a Linux command line.  The C++ source code for these

programs can be found in the sw/host directory.  These programs have two

dependencies: the ZipCPU load program depends upon libelf, and the ZipCPU

debugger depends upon the ncurses library.  If you have these two libraries,

your build should proceed without problems.  If now, you may get them simply

by ussuing a:

\begin{lstlisting}[language=bash]

% sudo apt-get install ncurses-dev libelf-dev texinfo

\end{lstlisting}

% TODO: Remove the dependency on ZIPD.

A make in the sw/host directory should build all of these support programs.

These include:

\begin{itemize}

\item {\tt wbregs}: a program to read and write addresses on the wishbone bus,

        and hence to test peripherals independent of the CPU.

\item {\tt netuart}: a program to convert the UART device provided by the board

        to a TCP/IP device that can be connected to anywhere.

\item {\tt wbsettime}: a simple program to set the time on the real-time clock

        core within the board.

\item {\tt dumpflash}: reads the current contents from the flash memory into a

        local file

\item {\tt wbprogram}: programs new configuations into the flash

\item {\tt netsetup}: reads and decodes the MDIO interface from the ethernet

        PHY controller

\item {\tt manping}: pings a computer using the ethernet packet interface.

        This program does not have any ARP handling, so while it will wait

        for a reply, the reply typically comes back in the form of an ARP

        request rather than the ping response.

\item {\tt zipload}: Loads a program onto the ZipCPU, adjusting flash, block

        RAM, and or SDRAM memory to do so.  May also start the program running

        if requested.

\item {\tt zipstate}: Returns information about whether or not the CPU is

        running, is running in user mode, is waiting for an interrupt,

        has halted, etc.

\item {\tt zipdbg}: a debugger with the capability to halt, reset and step

        the CPU, as well as to inspect the state of the CPU following any

        unexpected halt.

\end{itemize}

\section{Building the Verilator Simulation}

If you are at all interested in building the verilator simulation, you will

also need Verilator and GTKMM-3.0.  To get these, you may type:

\begin{lstlisting}[language=bash]

% sudo apt-get install verilator libgtkmm-3.0-dev

\end{lstlisting}

At this point, a {\tt make} in the {\tt rtl} directory, followed by a

{\tt make} in the {\tt bench/cpp} directory will build a Verilator simulation

named {\tt busmaster\_tb}.  You may run this program in place of {\tt netuart},

and then access the simulated Arty using the regular board support packages.

This simulation will use the TCP/IP port given in {\tt bench/cpp/port.h}, which

should be set identically to the port given in {\tt sw/host/port.h} used by

{\tt netuart}.

\section{Initially installing the core}

The OpenArty core may be installed onto the board via the Xilinx Hardware

Manager.  If properly set up, you should be able to open the hardware

manager after you build an initial bit stream, open the Arty, select the

toplevel bit file, and request Xilinx to load the file.

If you are successful, the four color LEDs will blank while the hardware

manager is loading the hardware, and then turn to varying intensities of red.

\section{Connecting the PMods}

The OpenArty project is designed to work with four PMods: PModUSBUART,

PModGPS, PModSD, and PModOLEDrgb.  These four provide the device with

serial port access, absolute time and position information, access to an

SD card, and the ability to control a small display.

If you do not have any of these devices, and wish to recover the logic used

by them, you may comment out the defines for {\tt GPS\_CLOCK},

{\tt SDCARD\_ACCESS}, and {\tt OLEDRGBACCESS} found in the {\tt rtl/busmaster.v}

file.  This will recover all but the logic used by the PModUSBUART and PModGPS

serial ports, while replacing the registers with read--only memory values of

zero.

The {\tt arty.xdc} file is designed so that these PMods can be connected as

shown in Fig.~\ref{fig:pmod-pic}.

\begin{figure}\begin{center}

\includegraphics[width=4in]{../gfx/openarty.eps}

\caption{Showing how the PMods are Connected}\label{fig:pmod-pic}

\end{center}\end{figure}

In this example, the PModOLED is connected to PMod port JB, and the PModSD is

connected to PMod port JD.  Both the PModGPS and the PModUSBUART are both

connected to port JC, with the GPS connected on top and the USBUART on the

bottom.

\section{Testing the peripherals}

OpenArty has been designed so that all of the peripherals live on a

memory--mapped wishbone bus.  This bus can be accessed, either by the ZipCPU

or by the host controller.  Because of this model, peripherals may be tested

and known to work before the CPU is ever turned on.  Two programs make this

possible: netuart and wbregs.  Other programs may be built upon this model,

as long as they access the bus using the interface outlined in devbus.h.

Of the two programs, netuart simply turns the USB serial port interface of

the device into a TCP/IP interface.  Netuart takes one argument, the

name of the serial port device which the Arty USB driver has created.  In

my case, this tends to be /dev/ttyUSB1, although it has been known to change

from time to time:

\begin{lstlisting}[language=bash]

% netuart /dev/ttyUSB1

\end{lstlisting}

All of the other board support files connect to the TCP/IP port generated

by netuart.  The port.h file is a compiled-in file, outlining where this

port can be found.  By default, netuart listens to port {\tt 6510} on

{\tt localhost}, but it can be configured to listen to any port.  The other

board support files will try to connect to netuart at the host and port

listed in port.h.  Hence, if properly configured, you should be able to

access your Arty to command it, configure it, reload it, etc., from anywhere

you have internet access--in my case, from anywhere in the house.

Once you run netuart, you should then be able to watch, as a part of the

standard output stream of netuart, all of the interaction with the board.

While this may be useful for debugging, it's not all that legible to the

user.   Lines that start with \hbox{``\#''} are lines from the device that are not

going to any client.  A common line you will see is \hbox{``\# 0''}.  This is

just the device saying that its command capability is idle.  Lines that start

with \hbox{``< ''} are commands going to the device, and lines starting with

\hbox{``> ''} are responses from the core.  So, at this point, run netuart and

wait a couple of seconds.  If you do not see a \hbox{``\# 0''} line, then try a

different serial port, check that your core is properly configured, etc.  Once

you do see a \hbox{``\# 0''} line, then you are ready for the next step.

The easiest way to test the peripherals is via the wbregs command.  This

command is similar to the ancient peek and poke commands.  It takes one

or two arguments.  If given one argument, it reads from that address on the

bus.  If given two arguments, it writes the value of the second argument to

the bus location given by the first argument.  Hence one argument peeks

at the memory bus, two arguments pokes a value onto the memory bus.

Perhaps an example will help.  At this point, try typing:

\begin{lstlisting}[language=bash]

% wbregs version

\end{lstlisting}

This should return and print a 32-bit hexadecimal value to your screen,

indicating the date of when you last ran make in the root directory before

building and installing your configuration into the device.  This can be

very useful to know what configuration you are running, and whether or not

you have made the changes you thought you had made.

You may have noticed that wbregs read from address 0x0100, but did so by name.

Most of the peripherals have names for their addresses.  The C language

names for these addresses can be found in regdefs.h, and a mapping to

wbregs address names can be found in regdefs.cpp.

Shall we try another?  Let's try adjusting the LEDs.  To turn all the LEDs off,

\begin{lstlisting}[language=bash]

% wbregs leds 0x0f0

\end{lstlisting}

To turn them all back on again,

\begin{lstlisting}[language=bash]

% wbregs leds 0x0ff

\end{lstlisting}

To turn the low order LED off without changing any others, write

\begin{lstlisting}[language=bash]

% wbregs leds 0x010

\end{lstlisting}

Having fun?  Try running the program startupex.sh from the sw directory.

This will set some LEDs and Color LEDs in a fun, startup--looking, pattern.

Ready to test the UART?   Using minicom, connect to the PModUSBUART.  It

should also be connected to a /dev/ttyUSBx serial port device.  If you aren't

sure, start minicom with:

\begin{lstlisting}[language=bash]

% minicom -D /dev/ttyUSB2

\end{lstlisting}

Then, configure minicom to use 115,200 Baud, 8-data bits, one stop bit, and

no parity.

Once you've done that, we can test it by sending a character across the UART

port:

\begin{lstlisting}[language=bash]

% wbregs tx 90

\end{lstlisting}

This should send a `Z' over the UART port.  Did you see a `Z' in minicom?

If not, did you set the baud rate right?  The UART is supposed to be set

for 115,200 Baud, 8N1 by default.  If not, you can set it to that by writing

wbregs setup 705.  The 705 comes from the clock rate, in Hz, divided by

115200.  By leaving other higher order bits at zero, this becomes the default

baud rate of an 8N1 serial port channel.

Another fun program to run is {\tt netsetup}.  This program takes no arguments,

and just reads and decodes the network registers via the MDIO port.  The

decoded result will be sent to the screen.

\section{Subsequent Core Updates}

The board support file {\tt wbprogram} can be used to write .bit or .bin

files to the flash, so that the core can be updated once an initial core

is installed and running.

Although wbprogram expects the filename to end in either '.bit' or '.bin',

this is primarily to keep a user from doing something they don't intend to

do.

The basic usage of the wbprogram command is:

\begin{lstlisting}[language=bash]

% wbprogram [@address] file

\end{lstlisting}

wbprogram then copies the file to the flash, starting at the Arty address

of {\tt @address}.  If no address is given, wbprogram writes the file at the

beginning of flash.

An example of how to do this can be found in the {\tt program.sh}.

{\tt program.sh} places the new configuration file into the alternate

configuration location.  (An alternate script, zprog.sh, places the new

configuration at the beginning of the flash, where the FPGA loader will look

for it upon power up.)  Once {\tt program.sh} places the new configuration

into flash, it then commands the FPGA via the ICAPE2 interface and an IPROG

command to reconfigure itself using this new configuration.  As a result, this

can be used to load subsequent configurations into the FLASH.

\section{Building the ZipCPU tool-chain}

At this point, you should have some confidence that your configuration and

hardware are working.  Therefore, let's transition to getting the ZipCPU

on the hardware up and running.

To do this, we'll start with getting a copy

of the ZipCPU toolchain and building it.  Pick a directory to work in, and

then issue:

\begin{lstlisting}[language=bash]

% git clone https://github.com/ZipCPU/zipcpu

\end{lstlisting}

to get a copy of the ZipCPU project, together with toolchain.  You'll also

need to double check that you have the pre-requisite packages to build this

tool chain, so on an Ubuntu~14 machine you would issue:

\begin{lstlisting}[language=bash]

% sudo apt-get install flex bison libbison-dev

% sudo apt-get install libgmp10 libgmp-dev libmpfr-dev libmpc-dev libelf-dev

% sudo apt-get install libisl-dev

\end{lstlisting}

Once these are all in place, you can then switch to the master ZipCPU

directory and type,

\begin{lstlisting}[language=bash]

% cd zipcpu; make

\end{lstlisting}

(you may need to issue the make command a couple of times \ldots)

This will build the GCC compiler for the ZipCPU from source.

It will also install this new compiler into the zipcpu/sw/install/cross-tools.

This new compiler will be called zip-gcc.

This will also build a copy of the binutils programs for the ZipCPU.  These

include the assembler, {\tt zip-gas}, linker, {\tt zip-ld}, disassembler,

{\tt zip-objdump}, and many more useful programs.

The next step to using this toolchain is to place it into your path.

\begin{lstlisting}[language=bash]

% export PATH=$PATH:$PWD/zipcpu/install/cross-tools/bin

\end{lstlisting}

Once the toolchain is in your path,

\begin{lstlisting}[language=bash]

% which zip-gcc

/home/.../zipcpu/sw/install/cross-tools/bin/zip-gcc

\end{lstlisting}

should return the location of where this toolchain exists in your path.

\section{Building your first ZipCPU program}

Several example programs for the OpenArty project can be found in the

{\tt sw/board} directory.  These can be used to test various peripherals from

the perspective of the CPU itself.

As a test of the build process, a good first progam to build would be

{\tt exstartup}.  This program is very similar to the {\tt startupex.sh} shell

script you tried earlier.  It simply plays with the color LEDs and some

on board timers.   Once that is finished, it goes into a loop controlling

both the normal and the color LEDs based upon the button state and the switch

settings.

To build {\tt exstartup}, simply type {\tt make exstartup} from the

{\tt sw/board} directory of the {\tt openarty} project.  (Don't forget to

include the ZipCPU toolchain into your path before you do this!)

\section{Loading a program}

Now that you have built your {\tt exstartup} program, it's time to load it

onto the board and start it up.  The {\tt zipload} program can be used to

do this.  {\tt zipload} can be found in the sw/host directory.  To load a

ZipCPU program into the Arty, just type {\tt zipload} and the program name,

such as {\tt exstartup} in this case.  To start the program immediately

after loading it, pass the `-r' option to {\tt zipload}.  In our case, you

would type:

\begin{lstlisting}[language=bash]

% zipload -r exstartup

\end{lstlisting}

Hopefully, you can see the {\tt exstartup} program now toggling the LED's.

Once the initial display stops, you can adjust the switches and press buttons

to see how that affects the result.

If you wish to restart the {\tt exstartup} program, or indeed to run another

program, you can just run {\tt zipload} again with the new program name.  This

will halt the previous program, and then load the new one into memory.  As

before, if you use the `-r' option, the program will be started automatically.

\section{Some other test programs}

If you have the PModUSBUART, you might wish to try running a ``Hello, World''

program.  This can be found in the hello.c file.  It prints ``Hello, World''

to the PModUSBUART once every ten seconds.  Had enough of it?  You can stop

the CPU by typing {\tt wbregs cpu 0x0400}.  This sends a halt command to the

debug register of the ZipCPU.  More information about this debug register, and

other things that can be done via the debug register, can be found in the

ZipCPU specification.

If you have both the PModUSBUART as well as the PModGPS, the {\tt gpsdump.c}

program can be used to forward the NMEA stream from the GPS to the USBUART.

This should give you some confidence that the PModGPS is working.

As a third test, {\tt oledtest.c} will initialize the OLEDrgb device and cause

it to display one of two images in an alternating fashion.

\chapter{Architecture}\label{ch:architecture}

My philosophy in peripherals is to keep them simple.  If there is a default

mode on the peripheral, setting that mode should not require turning any bits

on.  If a peripheral encounters an error condition, a bit may be turned on to

indicate this fact, otherwise status bits will be left in the off position.

\subsection{Bus Structure}

The OpenArty project contains four bus masters, three of them within the CPU.

These masters are the instruction fetch unit, the data read/write unit,

and the direct memory access peripheral within the ZipCPU, as well as an

external debug port which can be commanded from over the main UART port

connecting the Arty to its host.

There is also a second minor peripheral bus located within the ZipCPU

ZipSystem.  This bus provides access to a number of peripherals within the

ZipSystem, such as timers, counters, and the direct memory access controller.

This bus will also be used to configure the memory management unit once

integrated.  This bus is only visible to the CPU, and located starting at

address {\tt 0xc0000000}.

The ZipCPU debug port is also available on the bus.  This port, however, is

only visible to the external debug port.  It can be found at address

{\tt 0x08000000} for the control register, and {\tt 0x08000001} for the

data register.

Once the MMU has been integrated, it will be placed between the instruction

fetch unit, data read/write unit, and the rest of the peripheral bus.

The actual bus chosen for this design is the Wishbone Bus, based upon the

pipeline mode defined in the B4 specification.  All optional wires required

by this bus structure have been removed, such as the tag lines, the cycle

type identifier, the burst type, and so forth.  This was done to simplify

the logic within the core.

However, because of the complicated bus structure--particularly because of the

number of masters and slaves on the bus and the speed for which the bus is

defined, there are a number of delays and arbiters placed on the bus.  As a

result, the stall wire which is supposed to be depend upon combinational logic

only, has been registered at a number of locations.  What this means is that

there are a variety of delays as commands propagate through the bus structure.

Most of these are variable, in that they can be turned on or off at build time,

or even that the stall line may (or may not) be registered as configured.

All interactions between bus masters and any peripherals passes through the

interconnect, located in {\tt busmaster.v}.  This interconnect divides the

slaves into separate groups.  The first group of slaves are those for which the

bus is supposed to provide fast access to.  These are the DDR3 SDRAM, the

flash, the block RAM, and the network.  The next group of slaves will have their

acknowledgements delayed by an additional clock.  The final group of slaves

are those single register slaves whose results may be known ahead of any read,

and who only require one clock to access.  These are grouped together and

controlled from within {\tt fastio.v}.

Further information about the Wishbone bus structure found within this core

can be found either on the Wishbone datasheet (Ch.~\ref{ch:wishbone}), or in

the memory map table in the Registers chapter (Ch.~\ref{ch:registers}).

\subsection{DDR3 SDRAM}

{\em It is the intention of this project to use a completely open source

DDR3 SDRAM controller.  While the controller has been written, it has yet to

be successfully connected to the physical pins of the port.  Until that time,

the design is running using a Wishbone to AXI bus bridge.  Memory may still

be read or written, after an initial pipeline delay of roughly 27~clocks per

access, at one access per clock.}

{\em The open source SDRAM controller should be able to achieve a delay closer

to 9~clocks per access--once I figure out how to connect it to the PHY.}

\subsection{Flash}

\subsection{Block RAM}

The block RAM on this board has been arranged into one 32kW section.

Programs that use block RAM will run fastest using the block RAM, both for

instructions as well as for memory.

\subsection{Ethernet}

The ether net controller has been split into three parts.  The first part is

an area of packet memory.  This part is simple: it acts like memory.  The

receive memory is read only, whereas the transmit memory is both read and

write.  Packets received by the controller will be found in the receive memory,

packets transmitted must be in the transmit area of memory.  The octets

may be found in memory with the first octet in the most significant byte.

This is the easy part.

The format of the packets within this memory is a touch more interesting.

With no options turned on, the first 6~bytes are the destination MAC

address, the next 6~bytes will be the source MAC address, and the {\em next

4~bytes} will be the EtherType repeated twice.  This was done to align the

packet, and particularly the IP header, onto word boundaries.  If the hardware

CRC has been turned off, the packet must contain its own CRC as well as

ensuring that it has a minimum packet length (64 octets) when including that

CRC.

With all options turned on, however, things are a touch simpler.  The first

two words of the packet contain the destination MAC (for a transmit packet)

or the source MAC (for a received packet), followed by the two--octet

EtherType.  At this point the packet is word--aligned prior to the IP header.

Since broadcast packets are sent to a special destination MAC other than

our own, a flag in the command register will indicate this fact.

The second part of the controller is the MDIO interface.  This follows from

the specification, and can be used to toggle the LED's on the ethernet,

to force the ethernet into a particular mode, either 10M or 100M, to control

auto--negotiation of the speed, and more.  Reads or writes to MDIO memory

addresses will command reads or writes via the MDIO port from the FPGA to the

ethernet PHY.  As the PHY can only handle 16--bit words, only 16~bits will

ever be transferred as a result of any read/write command, the top 16~bits

are automatically set to zero.  Further details of this capability may be

found within the specification for the chip.

The MDIO interface may be ignored.  If ignored, the defaults within the

interface will naturally set up the network connection in full duplex mode (if

your hardware supports it), at the highest speed the network will support.

However, if you ignore this interface you may not know what problems you are

suffering from this interface, if any.  The {\tt netsetup} program has been

provided, among the host software, to help diagnose how the various MDIO

registers have been set, and what the status is that is being reported from

the PHY.

The third part of the controller is the packet command interface.  This

consists of two command registers, one for reading and one for writing.

Before doing anything with the network, it must first be taken out of

reset.  According to the specification for the network chip, this must

happen a minimum of one second after power up.  This may be done by simply

writing to the transmit command register with the reset bit turned off.

To send a packet, simply write the number of octets in the packet to the

transmit control register and set the GO bit ({\tt 0x04000}).  Other bits

in this control register can be used to turn off the hardware MAC generation

(and removal upon receive), the hardware CRC checking, and/or the hardware

IP header checksum validation (but not generation).  The GO bit will remain

high while the packet is being sent, and only transition to low once the

packet is away.  While the packet is being sent, a zero may be written to the

command register to cancel the packet--although this is not recommended.

Packets are automatically received without intervention.  Once a packet has been

received, the available bit will be set in the receive command register and

a receive packet interrupt will be generated.  The ethernet port will then

halt/stall until a user has reset the receive interface so that it may

receive the next packet.  Without clearing this interface, the receive port

will not accept further packets.  Other status bits in this interface are

used to indicate whether packets have been missed (because the interface was

busy), or thrown out due to some error such as a CRC error or a more general

error.\footnote{It should be possible to extend this interface so that further

packets may be read as long as the memory isn't yet full.  This is left as an

exercise to others.}

\subsection{SD Card}

\subsection{GPS Tracking}

\subsection{Configuration port}

The registers associated with the ICAPE2 port have been made accessible

to the core via the {\tt wbicapetwo} core.  More information about the meaning

of these registers can be found in Xilinx's ``7--Series FPGAs Configuration

User's Guide''.

Testing with the OpenArty board has tended to focus on the warmboot capability.

Using this capability, a user is able to command the FPGA to reload its

configuration.  In support of this, two configuration areas have been

defined within memory.  The first is the default configuration, found at

the beginning of the flash.  This configuration is sometimes called the ``golden

configuration'' within Xilinx's documentation because it is the configuration

that the Xilinx device will always start up from after a power on reset.  On

the OpenArty, a second configuration may immediately follow the first in flash.

Commanding the FPGA to reload it's configuration is as simple as

setting the WBSTAR (warm boot start address) register to the location of the

new configuration within the flash, and then writing a 15 (a.k.a. IPROG)

to the FPGA command register (offset 4 from the beginning of the ICAPE2

addresses).  Examples of doing this are found in the

{\tt sw/host/zprog.sh} and {\tt sw/host/program.sh} scripts.  The former

programs the default configuration and then switches to it,

This configuration capability makes it possible for a user to 1) reprogram

the flash with an experimental configuration in the second configuration

location, and 2) test the configuration without actually touching the board.

If the configuration doesn't work well enough to be communicated with, the

board may simply be powered down and it will come back up with the initial

or golden configuration.  If the golden configuration ever gets corrupted,

or loaded with a configuration that will not work, then the user will need to

reload the FPGA from the JTAG port.

\subsection{OLED}

\subsection{Real Time Clock}

The Arty board contains a real time clock core together with a companion

real time date/calendar core.  The clock core itself contains not only current

time, but also a stopwatch, seconds timer, and alarm.  The real time date core

can be used to maintain the current date.  The real--time clock core uses the

GPS PPS output, as schooled by the GPS tracking circuit, in order to synchronize

their subsecond timing to the GPS itself.  Further, the real--time clock core

then creates a synchronization wire for the real--time date core.

Neither of these cores exports its subsecond precision to the rest of the

design.  This must be done using either the internal GPS tracking wires, or

by reading the time information from the tracking test bench.

\subsection{LEDs}

The Arty board contains two sets of LEDs: a plain set of LEDs, and a colored

set of LEDs.

The plain set of LEDs is controlled simply from the LED register.  This register

can be used to turn these LEDs on and off, either individually or as a whole.

It has been designed for atomic access, so only one write to this register

is necessary to set any particular LED.

The color LEDs are slightly different.  Each color LED is supported by its

own register, which controls three pulse width modulation controllers.  Three

groups of eight bits within the color LED register control the PWM thresholds,

first for red, then green, and then in the lowest bits for blue.  These are

used to turn on and off the various color components of the LEDs.  Using this

method, there are $2^{24}$ different colors each of these LEDs may be set

to.

\subsection{Buttons}

\subsection{Switches}

\subsection{Startup counter}

A startup counter has been placed into the basic peripheral I/O area.  This

counter simply counts the clocks since startup.  Upon rollover, the high

order bit remains set.  This can be used to sequence the start up of components

within the design if so desired.

\subsection{GPS UART}

The GPS UART, debug control UART, as well as the auxilliary UART, are all

based upon the same underlying UART IP core, sometimes known as the WBUART32

core.  The setup register is defined within the documentation for that core,

and provides for a large baud rate selection, 5-8 data bits, 1-2 stop bits,

and several parity choices.  Within OpenArty, the GPS core is initialized

to 9.6~kBaud, 8 data bits, no parity, and one stop bit.

When a value is ready to be read from the GPS uart, the GPS interrupt line

will go high.  Once read, and only when read, will this interrupt line reset.

If the read is successful, only bits within the bottom eight will be set.

If a read is attempted when there is no data, when the UART is in a reset

condition, or when there has been a framing or parity error (were parity

to be turned on), the upper bits of the UART port will be set.

In a like manner, the GPS device can be written to.  Certain strings, if sent

to the UART, can be used to change the UARTs baud rate, its serial port

settings, or even its reporting interval.  As with the read port, the transmit

port will interrupt the CPU when it is idle.  Writing a character to this

port will reset the interrupt.  Setting bits other than the bottom eight may

result in a break condition being set on this port as well.

Interacting with a controller can therefore be somewhat tricky.  The

interrupt controller will trigger whenever the port is ready to be read from,

and will re--trigger every clock until the port has been read from.  At this

point, the interrupt controller may be reset.  If this is an auxilliary

interrupt controller, such as the bus interrupt controller or the ZipSystem's

auxiliary controller, the auxiliary controller will then need to be reset,

and the bit in the primary controller associated with the auxiliary controller

as well.  It is for this reason that the UARTs have been placed on the

primary controller only.

It should also be possible to use the DMA to read from (or write to) either

UART port.

\subsection{Auxilliary UART}

The Auxilliary UART has roughly the same structure as the GPS UART, save that

it's default configuration is for a 115,200~Baud configuration with 8~data bits,

no stop bits, and no parity.  Reads, writes, and interrupts are treated in

the same fashion.

\subsection{GPIO}

A General Purpose I/O controller has been placed within the design as well.

This controller can handle 16--generic input wires, and set 16--generic output

wires.  A single register is used to read both input and output wire values,

as well as to set output values when written to.

However, to use this controller, you will need to manually configure it

(i.e.~change the Verilog source) within the core, in order to wire the various

GPIO values up to a device of interest.  This was done for the simple reason

that wiring anything new up to the controller will require Verilog changes

anyway.  For this reason, the controller has no way of setting wires to high

impedence, or pulling them up or down.  Such control may be done within the

top level design if necessary.

This controller will set an interrupt if ever any of the input wires within

it are changed.  The interrupt may be cleared in the interrupt controller.

\subsection{Linker Script}

A linker script has been created to capture the memory structure needed by

a program.  This script may be found in {\tt sw/board/arty.ld}.  It is a

sample script, using it is not required.

The script defines three types of memory to the linker: flash, block RAM, and

SDRAM.  Programs using this script will naturally start in flash (acting as

a ROM memory).  A bootloader must then be used to copy, from flash, those

sections of the program that are to be placed in block RAM or SDRAM into

their particular memory locations.

The block RAM locations are reserved for the user kernel, and specifically for

any part of the code in the {\tt .kernel} section.  C attributes, or assembly

{\tt .section} commands, must be used to place items within this section.

A final symbol within this section, {\_top\_of\_stack}, is used so that the

initial boot loader knows what to set the initial kernel stack to.

The rest of the initial program's memory is placed into

SDRAM.\footnote{Hopefully,

I'll get a data cache running on the ZipCPU to speed this up.}  At the end,

a {\tt \_top\_of\_heap} symbol is set to reference the final location in the

setup.  This symbol can then be used as a starting point for a memory allocator.

An example bootloader is provided in {\tt sw/board} that can be linked with

any (bare metal, supervisor) program in order to properly load it into memory.

\chapter{Software}

\section{Directory Structure}

\section{Zip CPU Tool Chain}

\section{Bench Test Software}

\section{Host Software}

\begin{itemize}

\item {\tt readflash}: As I am loathe to remove anything from

        a device that came factory installed, the

        {\tt readflash} program reads the original installed

        configuration from the flash and dumps it to a file.

\item {\tt wbregs}: This program offers a capability very similar to the

        PEEK and POKE capability Apple user's may remember from before the

        days of Macintosh.  {\tt wbregs <address>} will read from the

        Wishbone bus the value at the given address.  Likewise

        {\tt wbregs <address> <value>} will write the given value into the

        given address.  While both address and value have the semantics of

        numbers acceptable to {\tt strtoul()}, the address can also be a named

        address.  Supported names can be found in {\tt regdefs.cpp}, and their

        register mapping in {\tt regdefs.h}.

\item {\tt ziprun}:

\item {\tt zipload}:

\end{itemize}

\section{Zip CPU Programs}

\begin{itemize}

\item {\tt ntpserver}:

\item {\tt goldenstart}:

\end{itemize}

\section{ZipOS}

\subsection{System Calls}

\begin{itemize}

\item {\tt int wait(unsigned event\_mask, int timeout)}

\item {\tt int clear(unsigned event\_mask, int timeout)}

\item {\tt void post(unsigned event\_mask)}

\item {\tt void yield(void) }

\item {\tt int read(int fid, void *buf, int len)}

\item {\tt int write(int fid, void *buf, int len)}

\item {\tt unsigned time(void) }

% \item SEMGET

% \item SEMPUT

\item {\tt void *malloc(void)}

\item {\tt void free(void *buf)}

% \item FORK

% \item opendir

% \item EXEC

% \item OPEN

\end{itemize}

\subsection{Scheduler}

\chapter{Operation}\label{ch:operation}

\chapter{Registers}\label{ch:registers}

There are several address regions within the {\tt OpenArty}, as shown in

Tbl.~\ref{tbl:memregions}.

\begin{table}[htbp]

\begin{center}\begin{tabular}{|p{2.25in}|p{0.6in}|p{0.35in}|p{2.0in}|}\hline

\rowcolor[gray]{0.85} Binary Address & \multicolumn{1}{c|}{Base} & \multicolumn{1}{c|}{Size} & Purpose \\\hline\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 0xxx xx--} & \scalebox{0.9}{\tt 0x00000400} & \hfill 128 & Peripheral I/O Control \\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 100y yx--} & \scalebox{0.9}{\tt 0x00000480} & \hfill 32& Debug scope control\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 1010 xx--} & \scalebox{0.9}{\tt 0x000004a0} & \hfill 16 & RTC control\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 1011 xx--} & \scalebox{0.9}{\tt 0x000004b0} & \hfill 16 & OLEDrgb control\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 1100 xx--} & \scalebox{0.9}{\tt 0x000004c0} & \hfill 16 & AUX UART\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 1101 xx--} & \scalebox{0.9}{\tt 0x000004d0} & \hfill 16 & GPS UART\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0100 1110 xx--} & \scalebox{0.9}{\tt 0x000004e0} & \hfill 16 & SDCard controller\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0101 0000 xx--} & \scalebox{0.9}{\tt 0x00000500} & \hfill 16 & GPS Clock loop control\\\hline

%\scalebox{0.8}{\tt 00 0000 0000 0000 0001 0101 0001 ----} & \scalebox{0.9}{\tt 0x00000148} & \hfill 16 & {\em Unused}\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0101 001x xx--} & \scalebox{0.9}{\tt 0x00000520} & \hfill 32 & GPS Testbench\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0101 010x xx--} & \scalebox{0.9}{\tt 0x00000540} & \hfill 32 & Network packet interface\\\hline

%\scalebox{0.8}{\tt 00 0000 0000 0000 0001 0101 011- ----} & \scalebox{0.9}{\tt 0x00000150} & \hfill 16 & {\em Unused}\\\hline

\scalebox{0.8}{\tt 00 0000 0000 0000 0000 0101 1xxx xx--} & \scalebox{0.9}{\tt 0x00000580} & \hfill 128 & Ethernet MIO configuration\\\hline