Subversion Repositories or1k

.\"

.\" Must use --  eqn -- with this one

.\"

.\" @(#)xdr.nts.ms      2.2 88/08/05 4.0 RPCSRC

.EQ

delim $$

.EN

.de BT

.if \\n%=1 .tl ''- % -''

..

.ND

.\" prevent excess underlining in nroff

.if n .fp 2 R

.OH 'External Data Representation: Sun Technical Notes''Page %'

.EH 'Page %''External Data Representation: Sun Technical Notes'

.if \\n%=1 .bp

.SH

\&External Data Representation: Sun Technical Notes

.IX XDR "Sun technical notes"

.LP

This chapter contains technical notes on Sun's implementation of the

External Data Representation (XDR) standard, a set of library routines

that allow a C programmer to describe arbitrary data structures in a

machinex-independent fashion.

For a formal specification of the XDR

standard, see the

.I "External Data Representation Standard: Protocol Specification".

XDR is the backbone of Sun's Remote Procedure Call package, in the

sense that data for remote procedure calls is transmitted using the

standard.  XDR library routines should be used to transmit data

that is accessed (read or written) by more than one type of machine.\**

.FS

.IX XDR "system routines"

For a compete specification of the system External Data Representation

routines, see the

.I xdr(3N)

manual page.

.FE

.LP

This chapter contains a short tutorial overview of the XDR library

routines, a guide to accessing currently available XDR streams, and

information on defining new streams and data types.  XDR was designed

to work across different languages, operating systems, and machine

architectures.  Most users (particularly RPC users) will only need

the information in the

.I "Number Filters",

.I "Floating Point Filters",

and

.I "Enumeration Filters"

sections.

Programmers wishing to implement RPC and XDR on new machines

will be interested in the rest of the chapter, as well as the

.I "External Data Representaiton Standard: Protocol Specification",

which will be their primary reference.

.SH

Note:

.I

.I rpcgen

can be used to write XDR routines even in cases where no RPC calls are

being made.

.LP

On Sun systems,

C programs that want to use XDR routines

must include the file

.I  ,

which contains all the necessary interfaces to the XDR system.

Since the C library

.I libc.a

contains all the XDR routines,

compile as normal.

.DS

example% \fBcc\0\fIprogram\fP.c\fI

.DE

.ne 3i

.NH 0

\&Justification

.IX XDR justification

.LP

Consider the following two programs,

.I writer :

.ie t .DS

.el .DS L

.ft CW

#include 

.sp.5

main()                  /* \fIwriter.c\fP */

{

        long i;

.sp.5

        for (i = 0; i < 8; i++) {

                if (fwrite((char *)&i, sizeof(i), 1, stdout) != 1) {

                        fprintf(stderr, "failed!\en");

                        exit(1);

                }

        }

        exit(0);

}

.DE

and

.I reader :

.ie t .DS

.el .DS L

.ft CW

#include 

.sp.5

main()                  /* \fIreader.c\fP */

{

        long i, j;

.sp.5

        for (j = 0; j < 8; j++) {

                if (fread((char *)&i, sizeof (i), 1, stdin) != 1) {

                        fprintf(stderr, "failed!\en");

                        exit(1);

                }

                printf("%ld ", i);

        }

        printf("\en");

        exit(0);

}

.DE

The two programs appear to be portable, because (a) they pass

.I lint

checking, and (b) they exhibit the same behavior when executed

on two different hardware architectures, a Sun and a VAX.

.LP

Piping the output of the

.I writer

program to the

.I reader

program gives identical results on a Sun or a VAX.

.DS

.ft CW

sun% \fBwriter | reader\fP

sun%

vax% \fBwriter | reader\fP

vax%

.DE

With the advent of local area networks and 4.2BSD came the concept

of \*Qnetwork pipes\*U \(em a process produces data on one machine,

and a second process consumes data on another machine.

A network pipe can be constructed with

.I writer

and

.I reader .

Here are the results if the first produces data on a Sun,

and the second consumes data on a VAX.

.DS

.ft CW

sun% \fBwriter | rsh vax reader\fP

117440512

sun%

.DE

Identical results can be obtained by executing

.I writer

on the VAX and

.I reader

on the Sun.  These results occur because the byte ordering

of long integers differs between the VAX and the Sun,

even though word size is the same.

Note that $16777216$ is $2 sup 24$ \(em

when four bytes are reversed, the 1 winds up in the 24th bit.

.LP

Whenever data is shared by two or more machine types, there is

a need for portable data.  Programs can be made data-portable by

replacing the

.I read()

and

.I write()

calls with calls to an XDR library routine

.I xdr_long() ,

a filter that knows the standard representation

of a long integer in its external form.

Here are the revised versions of

.I writer :

.ie t .DS

.el .DS L

.ft CW

#include 

#include        /* \fIxdr is a sub-library of rpc\fP */

.sp.5

main()          /* \fIwriter.c\fP */

{

        XDR xdrs;

        long i;

.sp.5

        xdrstdio_create(&xdrs, stdout, XDR_ENCODE);

        for (i = 0; i < 8; i++) {

                if (!xdr_long(&xdrs, &i)) {

                        fprintf(stderr, "failed!\en");

                        exit(1);

                }

        }

        exit(0);

}

.DE

and

.I reader :

.ie t .DS

.el .DS L

.ft CW

#include 

#include        /* \fIxdr is a sub-library of rpc\fP */

.sp.5

main()          /* \fIreader.c\fP */

{

        XDR xdrs;

        long i, j;

.sp.5

        xdrstdio_create(&xdrs, stdin, XDR_DECODE);

        for (j = 0; j < 8; j++) {

                if (!xdr_long(&xdrs, &i)) {

                        fprintf(stderr, "failed!\en");

                        exit(1);

                }

                printf("%ld ", i);

        }

        printf("\en");

        exit(0);

}

.DE

The new programs were executed on a Sun,

on a VAX, and from a Sun to a VAX;

the results are shown below.

.DS

.ft CW

sun% \fBwriter | reader\fP

sun%

vax% \fBwriter | reader\fP

vax%

sun% \fBwriter | rsh vax reader\fP

sun%

.DE

.SH

Note:

.I

.IX XDR "portable data"

Integers are just the tip of the portable-data iceberg.  Arbitrary

data structures present portability problems, particularly with

respect to alignment and pointers.  Alignment on word boundaries

may cause the size of a structure to vary from machine to machine.

And pointers, which are very convenient to use, have no meaning

outside the machine where they are defined.

.LP

.NH 1

\&A Canonical Standard

.IX XDR "canonical standard"

.LP

XDR's approach to standardizing data representations is

.I canonical .

That is, XDR defines a single byte order (Big Endian), a single

floating-point representation (IEEE), and so on.  Any program running on

any machine can use XDR to create portable data by translating its

local representation to the XDR standard representations; similarly, any

program running on any machine can read portable data by translating the

XDR standard representaions to its local equivalents.  The single standard

completely decouples programs that create or send portable data from those

that use or receive portable data.  The advent of a new machine or a new

language has no effect upon the community of existing portable data creators

and users.  A new machine joins this community by being \*Qtaught\*U how to

convert the standard representations and its local representations; the

local representations of other machines are irrelevant.  Conversely, to

existing programs running on other machines, the local representations of

the new machine are also irrelevant; such programs can immediately read

portable data produced by the new machine because such data conforms to the

canonical standards that they already understand.

.LP

There are strong precedents for XDR's canonical approach.  For example,

TCP/IP, UDP/IP, XNS, Ethernet, and, indeed, all protocols below layer five

of the ISO model, are canonical protocols.  The advantage of any canonical

approach is simplicity; in the case of XDR, a single set of conversion

routines is written once and is never touched again.  The canonical approach

has a disadvantage, but it is unimportant in real-world data transfer

applications.  Suppose two Little-Endian machines are transferring integers

according to the XDR standard.  The sending machine converts the integers

from Little-Endian byte order to XDR (Big-Endian) byte order; the receiving

machine performs the reverse conversion.  Because both machines observe the

same byte order, their conversions are unnecessary.  The point, however, is

not necessity, but cost as compared to the alternative.

.LP

The time spent converting to and from a canonical representation is

insignificant, especially in networking applications.  Most of the time

required to prepare a data structure for transfer is not spent in conversion

but in traversing the elements of the data structure.  To transmit a tree,

for example, each leaf must be visited and each element in a leaf record must

be copied to a buffer and aligned there; storage for the leaf may have to be

deallocated as well.  Similarly, to receive a tree, storage must be

allocated for each leaf, data must be moved from the buffer to the leaf and

properly aligned, and pointers must be constructed to link the leaves

together.  Every machine pays the cost of traversing and copying data

structures whether or not conversion is required.  In networking

applications, communications overhead\(emthe time required to move the data

down through the sender's protocol layers, across the network and up through

the receiver's protocol layers\(emdwarfs conversion overhead.

.NH 1

\&The XDR Library

.IX "XDR" "library"

.LP

The XDR library not only solves data portability problems, it also

allows you to write and read arbitrary C constructs in a consistent,

specified, well-documented manner.  Thus, it can make sense to use the

library even when the data is not shared among machines on a network.

.LP

The XDR library has filter routines for

strings (null-terminated arrays of bytes),

structures, unions, and arrays, to name a few.

Using more primitive routines,

you can write your own specific XDR routines

to describe arbitrary data structures,

including elements of arrays, arms of unions,

or objects pointed at from other structures.

The structures themselves may contain arrays of arbitrary elements,

or pointers to other structures.

.LP

Let's examine the two programs more closely.

There is a family of XDR stream creation routines

in which each member treats the stream of bits differently.

In our example, data is manipulated using standard I/O routines,

so we use

.I xdrstdio_create ().

.IX xdrstdio_create() "" "\fIxdrstdio_create()\fP"

The parameters to XDR stream creation routines

vary according to their function.

In our example,

.I xdrstdio_create()

takes a pointer to an XDR structure that it initializes,

a pointer to a

.I FILE

that the input or output is performed on, and the operation.

The operation may be

.I XDR_ENCODE

for serializing in the

.I writer

program, or

.I XDR_DECODE

for deserializing in the

.I reader

program.

.LP

Note: RPC users never need to create XDR streams;

the RPC system itself creates these streams,

which are then passed to the users.

.LP

The

.I xdr_long()

.IX xdr_long() "" "\fIxdr_long()\fP"

primitive is characteristic of most XDR library

primitives and all client XDR routines.

First, the routine returns

.I FALSE

(0) if it fails, and

.I TRUE

(1) if it succeeds.

Second, for each data type,

.I xxx ,

there is an associated XDR routine of the form:

.DS

.ft CW

xdr_xxx(xdrs, xp)

        XDR *xdrs;

        xxx *xp;

{

}

.DE

In our case,

.I xxx

is long, and the corresponding XDR routine is

a primitive,

.I xdr_long() .

The client could also define an arbitrary structure

.I xxx

in which case the client would also supply the routine

.I xdr_xxx (),

describing each field by calling XDR routines

of the appropriate type.

In all cases the first parameter,

.I xdrs

can be treated as an opaque handle,

and passed to the primitive routines.

.LP

XDR routines are direction independent;

that is, the same routines are called to serialize or deserialize data.

This feature is critical to software engineering of portable data.

The idea is to call the same routine for either operation \(em

this almost guarantees that serialized data can also be deserialized.

One routine is used by both producer and consumer of networked data.

This is implemented by always passing the address

of an object rather than the object itself \(em

only in the case of deserialization is the object modified.

This feature is not shown in our trivial example,

but its value becomes obvious when nontrivial data structures

are passed among machines.

If needed, the user can obtain the

direction of the XDR operation.

See the

.I "XDR Operation Directions"

section below for details.

.LP

Let's look at a slightly more complicated example.

Assume that a person's gross assets and liabilities

are to be exchanged among processes.

Also assume that these values are important enough

to warrant their own data type:

.ie t .DS

.el .DS L

.ft CW

struct gnumbers {

        long g_assets;

        long g_liabilities;

};

.DE

The corresponding XDR routine describing this structure would be:

.ie t .DS

.el .DS L

.ft CW

bool_t                  /* \fITRUE is success, FALSE is failure\fP */

xdr_gnumbers(xdrs, gp)

        XDR *xdrs;

        struct gnumbers *gp;

{

        if (xdr_long(xdrs, &gp->g_assets) &&

            xdr_long(xdrs, &gp->g_liabilities))

                return(TRUE);

        return(FALSE);

}

.DE

Note that the parameter

.I xdrs

is never inspected or modified;

it is only passed on to the subcomponent routines.

It is imperative to inspect the return value of each XDR routine call,

and to give up immediately and return

.I FALSE

if the subroutine fails.

.LP

This example also shows that the type

.I bool_t

is declared as an integer whose only values are

.I TRUE

(1) and

.I FALSE

(0).  This document uses the following definitions:

.ie t .DS

.el .DS L

.ft CW

#define bool_t  int

#define TRUE    1

#define FALSE   0

.DE

.LP

Keeping these conventions in mind,

.I xdr_gnumbers()

can be rewritten as follows:

.ie t .DS

.el .DS L

.ft CW

xdr_gnumbers(xdrs, gp)

        XDR *xdrs;

        struct gnumbers *gp;

{

        return(xdr_long(xdrs, &gp->g_assets) &&

                xdr_long(xdrs, &gp->g_liabilities));

}

.DE

This document uses both coding styles.

.NH 1

\&XDR Library Primitives

.IX "library primitives for XDR"

.IX XDR "library primitives"

.LP

This section gives a synopsis of each XDR primitive.

It starts with basic data types and moves on to constructed data types.

Finally, XDR utilities are discussed.

The interface to these primitives

and utilities is defined in the include file

.I  ,

automatically included by

.I  .

.NH 2

\&Number Filters

.IX "XDR library" "number filters"

.LP

The XDR library provides primitives to translate between numbers

and their corresponding external representations.

Primitives cover the set of numbers in:

.DS

.ft CW

[signed, unsigned] * [short, int, long]

.DE

.ne 2i

Specifically, the eight primitives are:

.DS

.ft CW

bool_t xdr_char(xdrs, cp)

        XDR *xdrs;

        char *cp;

.sp.5

bool_t xdr_u_char(xdrs, ucp)

        XDR *xdrs;

        unsigned char *ucp;

.sp.5

bool_t xdr_int(xdrs, ip)

        XDR *xdrs;

        int *ip;

.sp.5

bool_t xdr_u_int(xdrs, up)

        XDR *xdrs;

        unsigned *up;

.sp.5

bool_t xdr_long(xdrs, lip)

        XDR *xdrs;

        long *lip;

.sp.5

bool_t xdr_u_long(xdrs, lup)

        XDR *xdrs;

        u_long *lup;

.sp.5

bool_t xdr_short(xdrs, sip)

        XDR *xdrs;

        short *sip;

.sp.5

bool_t xdr_u_short(xdrs, sup)

        XDR *xdrs;

        u_short *sup;

.DE

The first parameter,

.I xdrs ,

is an XDR stream handle.

The second parameter is the address of the number

that provides data to the stream or receives data from it.

All routines return

.I TRUE

if they complete successfully, and

.I FALSE

otherwise.

.NH 2

\&Floating Point Filters

.IX "XDR library" "floating point filters"

.LP

The XDR library also provides primitive routines

for C's floating point types:

.DS

.ft CW

bool_t xdr_float(xdrs, fp)

        XDR *xdrs;

        float *fp;

.sp.5

bool_t xdr_double(xdrs, dp)

        XDR *xdrs;

        double *dp;

.DE

The first parameter,

.I xdrs

is an XDR stream handle.

The second parameter is the address

of the floating point number that provides data to the stream

or receives data from it.

Both routines return

.I TRUE

if they complete successfully, and

.I FALSE

otherwise.

.LP

Note: Since the numbers are represented in IEEE floating point,

routines may fail when decoding a valid IEEE representation

into a machine-specific representation, or vice-versa.

.NH 2

\&Enumeration Filters

.IX "XDR library" "enumeration filters"

.LP

The XDR library provides a primitive for generic enumerations.

The primitive assumes that a C

.I enum

has the same representation inside the machine as a C integer.

The boolean type is an important instance of the

.I enum .

The external representation of a boolean is always

.I TRUE

(1) or

.I FALSE

(0).

.DS

.ft CW

#define bool_t  int

#define FALSE   0

#define TRUE    1

.sp.5

#define enum_t int

.sp.5

bool_t xdr_enum(xdrs, ep)

        XDR *xdrs;

        enum_t *ep;

.sp.5

bool_t xdr_bool(xdrs, bp)

        XDR *xdrs;

        bool_t *bp;

.DE

The second parameters

.I ep

and

.I bp

are addresses of the associated type that provides data to, or

receives data from, the stream

.I xdrs .

.NH 2

\&No Data

.IX "XDR library" "no data"

.LP

Occasionally, an XDR routine must be supplied to the RPC system,

even when no data is passed or required.

The library provides such a routine:

.DS

.ft CW

bool_t xdr_void();  /* \fIalways returns TRUE\fP */

.DE

.NH 2

\&Constructed Data Type Filters

.IX "XDR library" "constructed data type filters"

.LP

Constructed or compound data type primitives

require more parameters and perform more complicated functions

then the primitives discussed above.

This section includes primitives for

strings, arrays, unions, and pointers to structures.

.LP

Constructed data type primitives may use memory management.

In many cases, memory is allocated when deserializing data with

.I XDR_DECODE

Therefore, the XDR package must provide means to deallocate memory.

This is done by an XDR operation,

.I XDR_FREE

To review, the three XDR directional operations are

.I XDR_ENCODE ,

.I XDR_DECODE

and

.I XDR_FREE .

.NH 3

\&Strings

.IX "XDR library" "strings"

.LP

In C, a string is defined as a sequence of bytes

terminated by a null byte,

which is not considered when calculating string length.

However, when a string is passed or manipulated,

a pointer to it is employed.

Therefore, the XDR library defines a string to be a

.I "char *"

and not a sequence of characters.

The external representation of a string is drastically different

from its internal representation.

Externally, strings are represented as

sequences of ASCII characters,

while internally, they are represented with character pointers.

Conversion between the two representations

is accomplished with the routine

.I xdr_string ():

.IX xdr_string() "" \fIxdr_string()\fP

.DS

.ft CW

bool_t xdr_string(xdrs, sp, maxlength)

        XDR *xdrs;

        char **sp;

        u_int maxlength;

.DE

The first parameter

.I xdrs

is the XDR stream handle.

The second parameter

.I sp

is a pointer to a string (type

.I "char **" .

The third parameter

.I maxlength

specifies the maximum number of bytes allowed during encoding or decoding.

its value is usually specified by a protocol.  For example, a protocol

specification may say that a file name may be no longer than 255 characters.

.LP

The routine returns

.I FALSE

if the number of characters exceeds

.I maxlength ,

and

.I TRUE

if it doesn't.

.SH

Keep

.I maxlength

small.  If it is too big you can blow the heap, since

.I xdr_string()

will call

.I malloc()

for space.

.LP

The behavior of

.I xdr_string()

.IX xdr_string() "" \fIxdr_string()\fP

is similar to the behavior of other routines

discussed in this section.  The direction

.I XDR_ENCODE

is easiest to understand.  The parameter

.I sp

points to a string of a certain length;

if the string does not exceed

.I maxlength ,

the bytes are serialized.

.LP

The effect of deserializing a string is subtle.

First the length of the incoming string is determined;

it must not exceed

.I maxlength .

Next

.I sp

is dereferenced; if the the value is

.I NULL ,

then a string of the appropriate length is allocated and

.I *sp

is set to this string.

If the original value of

.I *sp

is non-null, then the XDR package assumes

that a target area has been allocated,

which can hold strings no longer than

.I maxlength .

In either case, the string is decoded into the target area.

The routine then appends a null character to the string.

.LP

In the

.I XDR_FREE

operation, the string is obtained by dereferencing

.I sp .

If the string is not

.I NULL ,

it is freed and

.I *sp

is set to

.I NULL .

In this operation,

.I xdr_string()

ignores the

.I maxlength

parameter.

.NH 3

\&Byte Arrays

.IX "XDR library" "byte arrays"

.LP

Often variable-length arrays of bytes are preferable to strings.

Byte arrays differ from strings in the following three ways:

1) the length of the array (the byte count) is explicitly

located in an unsigned integer,

2) the byte sequence is not terminated by a null character, and

3) the external representation of the bytes is the same as their

internal representation.

The primitive

.I xdr_bytes()

.IX xdr_bytes() "" \fIxdr_bytes()\fP

converts between the internal and external

representations of byte arrays:

.DS

.ft CW

bool_t xdr_bytes(xdrs, bpp, lp, maxlength)

    XDR *xdrs;

    char **bpp;

    u_int *lp;

    u_int maxlength;

.DE

The usage of the first, second and fourth parameters

are identical to the first, second and third parameters of

.I xdr_string (),

respectively.

The length of the byte area is obtained by dereferencing

.I lp

when serializing;

.I *lp

is set to the byte length when deserializing.

.NH 3

\&Arrays

.IX "XDR library" "arrays"

.LP

The XDR library package provides a primitive

for handling arrays of arbitrary elements.

The

.I xdr_bytes()

routine treats a subset of generic arrays,

in which the size of array elements is known to be 1,

and the external description of each element is built-in.

The generic array primitive,

.I xdr_array() ,

.IX xdr_array() "" \fIxdr_array()\fP

requires parameters identical to those of

.I xdr_bytes()

plus two more:

the size of array elements,

and an XDR routine to handle each of the elements.

This routine is called to encode or decode

each element of the array.

.DS

.ft CW

bool_t

xdr_array(xdrs, ap, lp, maxlength, elementsiz, xdr_element)

    XDR *xdrs;

    char **ap;

    u_int *lp;

    u_int maxlength;

    u_int elementsiz;

    bool_t (*xdr_element)();

.DE

The parameter

.I ap

is the address of the pointer to the array.

If

.I *ap

is

.I NULL

when the array is being deserialized,

XDR allocates an array of the appropriate size and sets

.I *ap

to that array.

The element count of the array is obtained from

.I *lp

when the array is serialized;

.I *lp

is set to the array length when the array is deserialized.

The parameter

.I maxlength

is the maximum number of elements that the array is allowed to have;

.I elementsiz

is the byte size of each element of the array

(the C function

.I sizeof()

can be used to obtain this value).

The

.I xdr_element()

.IX xdr_element() "" \fIxdr_element()\fP

routine is called to serialize, deserialize, or free

each element of the array.

.br

.LP

Before defining more constructed data types, it is appropriate to

present three examples.

.LP

.I "Example A:"

.br

A user on a networked machine can be identified by

(a) the machine name, such as

.I krypton :

see the

.I gethostname

man page; (b) the user's UID: see the

.I geteuid

man page; and (c) the group numbers to which the user belongs:

see the

.I getgroups

man page.  A structure with this information and its associated

XDR routine could be coded like this:

.ie t .DS

.el .DS L

.ft CW

struct netuser {

    char    *nu_machinename;

    int     nu_uid;

    u_int   nu_glen;

    int     *nu_gids;

};

#define NLEN 255    /* \fImachine names < 256 chars\fP */

#define NGRPS 20    /* \fIuser can't be in > 20 groups\fP */

.sp.5

bool_t

xdr_netuser(xdrs, nup)

    XDR *xdrs;

    struct netuser *nup;

{

    return(xdr_string(xdrs, &nup->nu_machinename, NLEN) &&

        xdr_int(xdrs, &nup->nu_uid) &&

        xdr_array(xdrs, &nup->nu_gids, &nup->nu_glen,

        NGRPS, sizeof (int), xdr_int));

}

.DE

.LP

.I "Example B:"

.br

A party of network users could be implemented

as an array of

.I netuser

structure.

The declaration and its associated XDR routines

are as follows:

.ie t .DS

.el .DS L

.ft CW

struct party {

    u_int p_len;

    struct netuser *p_nusers;

};

#define PLEN 500    /* \fImax number of users in a party\fP */

.sp.5

bool_t

xdr_party(xdrs, pp)

    XDR *xdrs;

    struct party *pp;

{

    return(xdr_array(xdrs, &pp->p_nusers, &pp->p_len, PLEN,

        sizeof (struct netuser), xdr_netuser));

}

.DE

.LP

.I "Example C:"

.br

The well-known parameters to

.I main ,

.I argc

and

.I argv

can be combined into a structure.

An array of these structures can make up a history of commands.

The declarations and XDR routines might look like:

.ie t .DS

.el .DS L

.ft CW

struct cmd {

    u_int c_argc;

    char **c_argv;

};

#define ALEN 1000   /* \fIargs cannot be > 1000 chars\fP */

#define NARGC 100   /* \fIcommands cannot have > 100 args\fP */

struct history {

    u_int h_len;

    struct cmd *h_cmds;