OpenCores
URL https://opencores.org/ocsvn/openrisc/openrisc/trunk

Subversion Repositories openrisc

[/] [openrisc/] [trunk/] [rtos/] [ecos-2.0/] [doc/] [html/] [ref/] [kernel-smp.html] - Blame information for rev 588

Go to most recent revision | Details | Compare with Previous | View Log

Line No. Rev Author Line
1 28 unneback
<!-- Copyright (C) 2003 Red Hat, Inc.                                -->
2
<!-- This material may be distributed only subject to the terms      -->
3
<!-- and conditions set forth in the Open Publication License, v1.0  -->
4
<!-- or later (the latest version is presently available at          -->
5
<!-- http://www.opencontent.org/openpub/).                           -->
6
<!-- Distribution of the work or derivative of the work in any       -->
7
<!-- standard (paper) book form is prohibited unless prior           -->
8
<!-- permission is obtained from the copyright holder.               -->
9
<HTML
10
><HEAD
11
><TITLE
12
>SMP Support</TITLE
13
><meta name="MSSmartTagsPreventParsing" content="TRUE">
14
<META
15
NAME="GENERATOR"
16
CONTENT="Modular DocBook HTML Stylesheet Version 1.76b+
17
"><LINK
18
REL="HOME"
19
TITLE="eCos Reference Manual"
20
HREF="ecos-ref.html"><LINK
21
REL="UP"
22
TITLE="The eCos Kernel"
23
HREF="kernel.html"><LINK
24
REL="PREVIOUS"
25
TITLE="Kernel Overview"
26
HREF="kernel-overview.html"><LINK
27
REL="NEXT"
28
TITLE="Thread creation"
29
HREF="kernel-thread-create.html"></HEAD
30
><BODY
31
CLASS="REFENTRY"
32
BGCOLOR="#FFFFFF"
33
TEXT="#000000"
34
LINK="#0000FF"
35
VLINK="#840084"
36
ALINK="#0000FF"
37
><DIV
38
CLASS="NAVHEADER"
39
><TABLE
40
SUMMARY="Header navigation table"
41
WIDTH="100%"
42
BORDER="0"
43
CELLPADDING="0"
44
CELLSPACING="0"
45
><TR
46
><TH
47
COLSPAN="3"
48
ALIGN="center"
49
>eCos Reference Manual</TH
50
></TR
51
><TR
52
><TD
53
WIDTH="10%"
54
ALIGN="left"
55
VALIGN="bottom"
56
><A
57
HREF="kernel-overview.html"
58
ACCESSKEY="P"
59
>Prev</A
60
></TD
61
><TD
62
WIDTH="80%"
63
ALIGN="center"
64
VALIGN="bottom"
65
></TD
66
><TD
67
WIDTH="10%"
68
ALIGN="right"
69
VALIGN="bottom"
70
><A
71
HREF="kernel-thread-create.html"
72
ACCESSKEY="N"
73
>Next</A
74
></TD
75
></TR
76
></TABLE
77
><HR
78
ALIGN="LEFT"
79
WIDTH="100%"></DIV
80
><H1
81
><A
82
NAME="KERNEL-SMP">SMP Support</H1
83
><DIV
84
CLASS="REFNAMEDIV"
85
><A
86
NAME="AEN206"
87
></A
88
><H2
89
>Name</H2
90
>SMP&nbsp;--&nbsp;Support Symmetric Multiprocessing Systems</DIV
91
><DIV
92
CLASS="REFSECT1"
93
><A
94
NAME="KERNEL-SMP-DESCRIPTION"
95
></A
96
><H2
97
>Description</H2
98
><P
99
>eCos contains support for limited Symmetric Multi-Processing (SMP).
100
This is only available on selected architectures and platforms.
101
The implementation has a number of restrictions on the kind of
102
hardware supported. These are described in <A
103
HREF="hal-smp-support.html"
104
>the Section called <I
105
>SMP Support</I
106
> in Chapter 9</A
107
>.
108
    </P
109
><P
110
>The following sections describe the changes that have been made to the
111
eCos kernel to support SMP operation.
112
    </P
113
></DIV
114
><DIV
115
CLASS="REFSECT1"
116
><A
117
NAME="KERNEL-SMP-STARTUP"
118
></A
119
><H2
120
>System Startup</H2
121
><P
122
>The system startup sequence needs to be somewhat different on an SMP
123
system, although this is largely transparent to application code. The
124
main startup takes place on only one CPU, called the primary CPU. All
125
other CPUs, the secondary CPUs, are either placed in suspended state
126
at reset, or are captured by the HAL and put into a spin as they start
127
up. The primary CPU is responsible for copying the DATA segment and
128
zeroing the BSS (if required), calling HAL variant and platform
129
initialization routines and invoking constructors. It then calls
130
<TT
131
CLASS="FUNCTION"
132
>cyg_start</TT
133
> to enter the application. The
134
application may then create extra threads and other objects.
135
      </P
136
><P
137
>It is only when the application calls
138
<TT
139
CLASS="FUNCTION"
140
>cyg_scheduler_start</TT
141
> that the secondary CPUs are
142
initialized. This routine scans the list of available secondary CPUs
143
and invokes <TT
144
CLASS="FUNCTION"
145
>HAL_SMP_CPU_START</TT
146
> to start each
147
CPU. Finally it calls an internal function
148
<TT
149
CLASS="FUNCTION"
150
>Cyg_Scheduler::start_cpu</TT
151
> to enter the scheduler
152
for the primary CPU.
153
      </P
154
><P
155
>Each secondary CPU starts in the HAL, where it completes any per-CPU
156
initialization before calling into the kernel at
157
<TT
158
CLASS="FUNCTION"
159
>cyg_kernel_cpu_startup</TT
160
>. Here it claims the
161
scheduler lock and calls
162
<TT
163
CLASS="FUNCTION"
164
>Cyg_Scheduler::start_cpu</TT
165
>.
166
      </P
167
><P
168
><TT
169
CLASS="FUNCTION"
170
>Cyg_Scheduler::start_cpu</TT
171
> is common to both the
172
primary and secondary CPUs. The first thing this code does is to
173
install an interrupt object for this CPU's inter-CPU interrupt. From
174
this point on the code is the same as for the single CPU case: an
175
initial thread is chosen and entered.
176
      </P
177
><P
178
>From this point on the CPUs are all equal, eCos makes no further
179
distinction between the primary and secondary CPUs. However, the
180
hardware may still distinguish between them as far as interrupt
181
delivery is concerned.
182
      </P
183
></DIV
184
><DIV
185
CLASS="REFSECT1"
186
><A
187
NAME="KERNEL-SMP-SCHEDULING"
188
></A
189
><H2
190
>Scheduling</H2
191
><P
192
>To function correctly an operating system kernel must protect its
193
vital data structures, such as the run queues, from concurrent
194
access. In a single CPU system the only concurrent activities to worry
195
about are asynchronous interrupts. The kernel can easily guard its
196
data structures against these by disabling interrupts. However, in a
197
multi-CPU system, this is inadequate since it does not block access by
198
other CPUs.
199
      </P
200
><P
201
>The eCos kernel protects its vital data structures using the scheduler
202
lock. In single CPU systems this is a simple counter that is
203
atomically incremented to acquire the lock and decremented to release
204
it. If the lock is decremented to zero then the scheduler may be
205
invoked to choose a different thread to run. Because interrupts may
206
continue to be serviced while the scheduler lock is claimed, ISRs are
207
not allowed to access kernel data structures, or call kernel routines
208
that can. Instead all such operations are deferred to an associated
209
DSR routine that is run during the lock release operation, when the
210
data structures are in a consistent state.
211
      </P
212
><P
213
>By choosing a kernel locking mechanism that does not rely on interrupt
214
manipulation to protect data structures, it is easier to convert eCos
215
to SMP than would otherwise be the case. The principal change needed to
216
make eCos SMP-safe is to convert the scheduler lock into a nestable
217
spin lock. This is done by adding a spinlock and a CPU id to the
218
original counter.
219
      </P
220
><P
221
>The algorithm for acquiring the scheduler lock is very simple. If the
222
scheduler lock's CPU id matches the current CPU then it can just increment
223
the counter and continue. If it does not match, the CPU must spin on
224
the spinlock, after which it may increment the counter and store its
225
own identity in the CPU id.
226
      </P
227
><P
228
>To release the lock, the counter is decremented. If it goes to zero
229
the CPU id value must be set to NONE and the spinlock cleared.
230
      </P
231
><P
232
>To protect these sequences against interrupts, they must be performed
233
with interrupts disabled. However, since these are very short code
234
sequences, they will not have an adverse effect on the interrupt
235
latency.
236
      </P
237
><P
238
>Beyond converting the scheduler lock, further preparing the kernel for
239
SMP is a relatively minor matter. The main changes are to convert
240
various scalar housekeeping variables into arrays indexed by CPU
241
id. These include the current thread pointer, the need_reschedule
242
flag and the timeslice counter.
243
      </P
244
><P
245
>At present only the Multi-Level Queue (MLQ) scheduler is capable of
246
supporting SMP configurations. The main change made to this scheduler
247
is to cope with having several threads in execution at the same
248
time. Running threads are marked with the CPU that they are executing on.
249
When scheduling a thread, the scheduler skips past any running threads
250
until it finds a thread that is pending. While not a constant-time
251
algorithm, as in the single CPU case, this is still deterministic,
252
since the worst case time is bounded by the number of CPUs in the
253
system.
254
      </P
255
><P
256
>A second change to the scheduler is in the code used to decide when
257
the scheduler should be called to choose a new thread. The scheduler
258
attempts to keep the <SPAN
259
CLASS="PROPERTY"
260
>n</SPAN
261
> CPUs running the
262
<SPAN
263
CLASS="PROPERTY"
264
>n</SPAN
265
> highest priority threads. Since an event or
266
interrupt on one CPU may require a reschedule on another CPU, there
267
must be a mechanism for deciding this. The algorithm currently
268
implemented is very simple. Given a thread that has just been awakened
269
(or had its priority changed), the scheduler scans the CPUs, starting
270
with the one it is currently running on, for a current thread that is
271
of lower priority than the new one. If one is found then a reschedule
272
interrupt is sent to that CPU and the scan continues, but now using
273
the current thread of the rescheduled CPU as the candidate thread. In
274
this way the new thread gets to run as quickly as possible, hopefully
275
on the current CPU, and the remaining CPUs will pick up the remaining
276
highest priority threads as a consequence of processing the reschedule
277
interrupt.
278
      </P
279
><P
280
>The final change to the scheduler is in the handling of
281
timeslicing. Only one CPU receives timer interrupts, although all CPUs
282
must handle timeslicing. To make this work, the CPU that receives the
283
timer interrupt decrements the timeslice counter for all CPUs, not
284
just its own. If the counter for a CPU reaches zero, then it sends a
285
timeslice interrupt to that CPU. On receiving the interrupt the
286
destination CPU enters the scheduler and looks for another thread at
287
the same priority to run. This is somewhat more efficient than
288
distributing clock ticks to all CPUs, since the interrupt is only
289
needed when a timeslice occurs.
290
      </P
291
><P
292
>All existing synchronization mechanisms work as before in an SMP
293
system. Additional synchronization mechanisms have been added to
294
provide explicit synchronization for SMP, in the form of
295
<A
296
HREF="kernel-spinlocks.html"
297
>spinlocks</A
298
>.
299
      </P
300
></DIV
301
><DIV
302
CLASS="REFSECT1"
303
><A
304
NAME="KERNEL-SMP-INTERRUPTS"
305
></A
306
><H2
307
>SMP Interrupt Handling</H2
308
><P
309
>The main area where the SMP nature of a system requires special
310
attention is in device drivers and especially interrupt handling. It
311
is quite possible for the ISR, DSR and thread components of a device
312
driver to execute on different CPUs. For this reason it is much more
313
important that SMP-capable device drivers use the interrupt-related
314
functions correctly. Typically a device driver would use the driver
315
API rather than call the kernel directly, but it is unlikely that
316
anybody would attempt to use a multiprocessor system without the
317
kernel package.
318
      </P
319
><P
320
>Two new functions have been added to the Kernel API
321
to do <A
322
HREF="kernel-interrupts.html#KERNEL-INTERRUPTS-SMP"
323
>interrupt
324
routing</A
325
>: <TT
326
CLASS="FUNCTION"
327
>cyg_interrupt_set_cpu</TT
328
> and
329
<TT
330
CLASS="FUNCTION"
331
>cyg_interrupt_get_cpu</TT
332
>. Although not currently
333
supported, special values for the cpu argument may be used in future
334
to indicate that the interrupt is being routed dynamically or is
335
CPU-local. Once a vector has been routed to a new CPU, all other
336
interrupt masking and configuration operations are relative to that
337
CPU, where relevant.
338
      </P
339
><P
340
>There are more details of how interrupts should be handled in SMP
341
systems in <A
342
HREF="devapi-smp-support.html"
343
>the Section called <I
344
>SMP Support</I
345
> in Chapter 18</A
346
>.
347
      </P
348
></DIV
349
><DIV
350
CLASS="NAVFOOTER"
351
><HR
352
ALIGN="LEFT"
353
WIDTH="100%"><TABLE
354
SUMMARY="Footer navigation table"
355
WIDTH="100%"
356
BORDER="0"
357
CELLPADDING="0"
358
CELLSPACING="0"
359
><TR
360
><TD
361
WIDTH="33%"
362
ALIGN="left"
363
VALIGN="top"
364
><A
365
HREF="kernel-overview.html"
366
ACCESSKEY="P"
367
>Prev</A
368
></TD
369
><TD
370
WIDTH="34%"
371
ALIGN="center"
372
VALIGN="top"
373
><A
374
HREF="ecos-ref.html"
375
ACCESSKEY="H"
376
>Home</A
377
></TD
378
><TD
379
WIDTH="33%"
380
ALIGN="right"
381
VALIGN="top"
382
><A
383
HREF="kernel-thread-create.html"
384
ACCESSKEY="N"
385
>Next</A
386
></TD
387
></TR
388
><TR
389
><TD
390
WIDTH="33%"
391
ALIGN="left"
392
VALIGN="top"
393
>Kernel Overview</TD
394
><TD
395
WIDTH="34%"
396
ALIGN="center"
397
VALIGN="top"
398
><A
399
HREF="kernel.html"
400
ACCESSKEY="U"
401
>Up</A
402
></TD
403
><TD
404
WIDTH="33%"
405
ALIGN="right"
406
VALIGN="top"
407
>Thread creation</TD
408
></TR
409
></TABLE
410
></DIV
411
></BODY
412
></HTML
413
>

powered by: WebSVN 2.1.0

© copyright 1999-2024 OpenCores.org, equivalent to Oliscience, all rights reserved. OpenCores®, registered trademark.