Multiclock designs in FPGA

Multiple, independent clocks are ubiquitous in system-on-chip (SoC) design. Most SoC devices have multiple interfaces, some following standards that use very different clock frequencies.

Many modern serial interfaces are inherently asynchronous from the rest of the chip; some actually derive their clocks directly from the incoming data streams. There is also a trend toward designing major sub-blocks of SoCs to run on independent clocks to ease the problem of clock skew across large chips.

For all of these reasons, designers working on SoC projects are virtually certain to encounter multiple clocks and to be faced with the design of logic interconnecting two portions of the chip running on independent clocks. Each such portion is known as a clock domain. The interface between logic on different clocks is called a clock domain crossing or clock domain boundary. The proper handing of signals across clock domain boundaries is critical for successful SoC design.

Problem #1: Meta-stability

The first multi-clock problem that designers must consider is that of meta-stability as signals pass from one clock domain to another. Most designers understand that meta-stability is a real problem in real circuits; the modern abstractions of RTL design and static timing analysis can't entirely shield designers from having to worry about the underlying physics.

Whenever a signal enters a clocked circuit element, such as a flip-flop, too close to the clock, there is the potential for meta-stability. When this happens, the flip-flop may not immediately settle to a known value. It is critical that the output signal from the flip-flop not be used until it has settled.

On a truly asynchronous clock boundary, the receiving domain's clock is used to capture each signal from the driving domain in a flip-flop. Because there is no defined temporal relationship between the clock and the signal, it is entirely possible that they could transition at the same time. Whenever this happens, there is a possibility of meta-stability in the receiving clock domain.

This is not just a theoretical potential. GHz-rate chips with clocking design errors can exhibit the effects of meta-stability quite quickly when running in real systems. These effects typically include loss of critical handshake signals between clocks domains, and corruption of multi-bit data"serious problems that are highly likely to require a chip re-spin.

Most designers also know that the textbook solution to meta-stability is using two levels of flip-flops on each signal crossing a clock domain boundary. Even if the first flip-flop does become meta-stable, there is an extremely high likelihood that the signal will settle by the time that it passes through the second level. The double-level flip-flop structure is often called a synchronizer, and designers commonly speak of synchronizing signals across clock domains.

Problem #2: Reset synchronization

Improper synchronization of reset signals is a related problem in multi-clock designs. Designers sometimes forget that reset signals are subject to meta-stability and must be protected by synchronizers. Generally, the entire SoC can be reset by a single signal, which therefore must propagate to all clocked elements in all clock domains.

There is no need for synchronization on the activation edge of reset, since by definition all state elements are reset to initial values, and the reset signal will generally be held active for enough cycles to allow any meta-stability to settle out.

However, when reset is deactivated it must be properly synchronized so that flip-flops do not become meta-stable as they come out of the reset state. This can be accomplished by having a separate synchronizer for the reset signal as it enters into each clock domain in the design.

Problem #3: Glitch elimination across clock domain boundaries

Although synchronization eliminates meta-stability for all practical purposes, it is not sufficient to fix other types of multi-clock design errors. The third common error in multi-clock design is allowing glitch propagation from the driving clock domain to the receiving clock domain.

Since a flip-flop in the receiving clock domain can sample a signal from the driving domain at any point in time, even a very narrow glitch might be captured and treated as a valid value. Because of the variable nature of the clock relationships, this glitch potential cannot be detected by static timing analysis.

The most common way to avoid this type of problem is to mandate that each signal entering the synchronizer come directly from a flip-flop in the driving clock domain. This effectively defines a three-stage synchronizer, with the first flip-flip driven by the driving domain clock and the remaining two stages driven by the receiving clock. This approach could be enforced by design reviews, although automatic analysis of the design and its clocking structures is more reliable.

Problem #4: Inadequate hold times for receiving clock domains

The fourth category of multi-clock design problems is data instability from failure to observe hold times as signals pass from one clock domain into another. A signal passing from a fast clock domain into a slower domain must be stable for multiple cycles in the driving domain to ensure that the slower clock in the receiving domain will not miss a transition entirely.

If such a transition is missed, data loss will occur. For example, a signal passing from a 33-MHz PCI domain to a 12-MHZ USB domain must be held stable for three cycles to ensure that it will be received.

Hold times are also important when data signals are passed across a clock domain boundary under the logical protection of a control signal.

Problem #5: Loss of correlation among signals crossing clock domains

The final category of multi-clock design problems is the least intuitive: correlated signals that can fall out of correlation as they cross a clock domain boundary. This can lead to highly unpredictable behavior such as data loss and corruption of arithmetic results.

These non-intuitive problems can be very hard to detect by visual inspection, especially when the design is expressed in RTL form, and so are often missed in design reviews. Strict design rules or effective automatic analysis must be used to these problems.

There are several different ways that loss of correlation can occur for a wide range of correlated signals, including:

* Multiple bits of a bus
* Multiple copies of a single signal
* Handshake signals, such as request and acknowledge
* Apparently unrelated signals that merge into the same logic in the receiving clock domain

Whenever a signal goes meta-stable, a synchronizer settles it down but cannot guarantee a precise number of cycles before the valid signal is available in the receiving clock domain. Therefore, anytime a multi-bit signal has each bit independently synchronized, the bits may arrive in the receiving clock domain skewed with respect to one another.

Even when meta-stability does not occur, any pair of bits can get out of synchronization if routing differences and electrical effects cause the two bits to have different delays before reaching their respective synchronizers.

It is possible for one synchronizer to sample its input and capture a signal change before the other synchronizer captures the change, at which point the two signals will be skewed by a cycle and therefore no longer correlated.

Loss of correlation can also occur when two apparently independent signals are synchronized separately across the clock boundary but ultimately fed into the same logic in the receiving clock domain. This scenario, sometimes called reconvergence, is especially difficult to detect by manual inspection of the synchronization schemes during design review.

A solution: Clock intent verification
Although the challenges for multi-clock design are indeed daunting, it is fortunate that all of the problems described thus far are amenable to automatic analysis. While not all problems can be definitively identified, it is possible to provide detailed warnings about suspicious logic so that the designers can check to see whether or not the logic is correct. Clock intent verification (CIV) is the process of analyzing the clocking structure, reporting errors and warnings, and then using exhaustive formal verification to verify issues that cannot be proved otherwise.

Any CIV solution must address each of the five problem areas in multi-clock design. The process starts with a structural analysis of the design in RTL form, tracing all clocks and resets in the design from the flop-flops and other state elements. This process can automatically identify:

* All asynchronous clock domains in the design
* All control and data signals crossing between clock domains
* Any domain-crossing signals that have missing or incorrect synchronizers
* Any synchronizers that have the potential for glitches on their inputs
* Any signals that have fan-outs to multiple synchronizers
* Any independently synchronized signals that reconverge in the receiving clock domain
* Any clock domains whose reset signals are not properly synchronized
* Any gated or derived clocks with glitch potential

If properly implemented, this structural analysis can be performed quickly enough to run on an entire SoC in a matter of minutes. Minimal designer input is required, although the analysis can benefit if a simple description of input clock behavior is provided.

With this information, the analysis can be extremely accurate so that spurious errors and warnings are not reported. The designers should see only messages that report real design problems or provide useful feedback about the design (such as a list of all signals that are properly synchronized).

Although most types of multi-clock design problems can be detected with CIV structural analysis, a few categories cannot. Verifying proper handling of multi-bit signals and checking for data stability across clock domains requires the exhaustive power of formal verification for thorough analysis.

Since formal verification requires assertions about the design, a CIV solution must be able to generate three categories of assertions to represent the desired behavior of signals crossing clock domains:

* Assertions that multi-bit signals crossing clock domains must always be gray-coded (only one bit changing at a time); this avoids loss of correlation.
* Assertions that signals with synchronizers remain stable for enough cycles to be detected in the receiving clock domain; this avoids data loss.
* Assertions that signals without synchronizers (such as data buses) are held stable during the time that the signals are sampled in the receiving clock domain; this also avoids data loss.

Formal verification tries to prove that these assertions can never be violated under any scenarios, or tries to generate counter-examples showing how the assertions can be violated.

Whenever a proof is found, designer confidence is increased since they know that there is no possible way that a particular clock domain crossing error can occur. Whenever a counter-example is found, formal verification has detected a real bug in the design that could result in chip failure if not resolved. The formal tools must provide enough information for the designers to resolve the error and then re-run to see if a proof can be obtained to erase any lingering worries about possible errors.

Although formal verification is an exhaustive technology, in practice not all assertions can be proven in complex SoC designs. For this reason, it is valuable if a CIV solution can also generate the assertions in a form suitable for simulation. If a proof cannot be found but the assertion runs with no violations throughout a typical regression test suite of many millions or even billions of cycles, then the designers still gain an important measure of confidence in the correctness of the multi-clock design.