Adding C Functions to Shading Language with DSOs

1. Introduction

RenderMan Shading Language has always had a rich library of built-in functions (sometimes called "shadeops"), already known to the SL compiler and implemented as part of the runtime shader interpreter in the renderer. This built-in function library included math operations (sin, sqrt, etc.), vector and matrix operations, coordinate transformations, etc. New and useful built-in functions have been steadily added with each new PRMan release.

It has also been possible to write SL functions in Shading Language itself. The improvements to SL function semantics and compiler robustness was especially increased in the new SL compiler introduced with PRMan 3.7, nshader. However, defining functions in SL itself has several limitations.

This release of PRMan allows you to write new built-in SL functions in 'C' or 'C++'. Such functions overcome many of the limitations of SL-defined functions.

Writing new shadeops in C and linking them as DSO's has many advantages over writing functions in SL, including:

• The resulting object code from a DSO shadeop is shared among all its uses in a renderer. In contrast, compiled shader function code is inlined every time the function is called, and thus is not shared among its uses, let along among separate shaders that call the same function.
• DSO shadeops are compiled to optimized machine code, versus shader functions which are interpreted at runtime. While PRMan has a very efficient interpreter, it is definitely slower than native machine code.
• DSO shadeops can call library functions from the standard C library or from other third party libraries.
• Whereas functions implemented in SL are restricted to operations and data structures available in the Shading Language, DSO shadeops can do anything you might normally do in a C program. Examples include creating complex data structures or reading external files (other than textures and shadows). For example, implementing an alternative noise() function, which needs a stored table to be efficient, would be exceptionally difficult in SL, but very easy as a DSO shadeop.

DSO shadeops have several limitations that you should be aware of:

• DSO shadeops only have access to information passed to them as parameters. They have no knowledge of "global shaders variables" such as P, parameters to the shader, or any other renderer state. If you need to access global variables or shader parameters or locals, you must pass them as parameters.
• DSO shadeops act as strictly point processes. They posess no knowledge of the topology of the surface, derivatives, or the nature of surface grids (in the case of a REYES renderer like PRMan). If you want to take derivatives, for example, you need to take them in the shader and pass them as parameters to your DSO shadeop.
• DSO shadeops cannot call other builtin shadeops or any other internal entry points to the renderer itself.

The C File

You can compile your DSO's either with a 'C' or 'C++' compiler, but you must use 'C' linkage -- i.e. if you use C++, you need to use extern "C" to guarantee C-style linkage.

We have provided a header, shadeop.h, which contains some macros which are handy for writing your own SL builtin functions.

Methods, Initializers, Shutdown functions

Each new shadeop has three C function components: an init function, a method, and a shutdown function. They have the following C prototypes:

void *init(int ctx, void *texturectx);
int   method(void *initdata, int argc, void **argv);
void  shutdown(void *initdata);

The optional init function is called prior to the first use of the method, and may be used to initialize variables, allocate memory, build data structures, etc. The init function takes two arguments: an integer signifying a unique integer ID for the thread, and a void* which is a blind handle to a texturing context for the thread. If your shadeop method routines will need this data, you should stash it somewhere in the data that you allocate and return from the initializer routine; however, they are not required to use these parameters in any way. The intended use of the texture context pointer is that it will be required to pass in to a future API entry point which will allow your DSO shadeops to request filtered texture lookups. (This may be introduced in future releases of PRMan.)

The init function returns a void *, which may return any data structures that you allocate during initialization. An init function may return NULL, or you may not have an init function at all, if you do not need explicit initialization. The init function is generally called only once per invocation of the renderer, but may be called once per thread in the case of multithreaded renderers. Note that if multiple DSO shadeops in the same .so file all use the same named initializer routine, then the initializer will be called only the first time that any of those shadeops is invoked. Upon further calls to any of the shadeops which share an initializer, they will all be passed pointers to the data block returned by the first initializer. In other words, all of the shadeops which share the same initializer will also share the same per-thread data store.

The method is the actual implementation of the shadeop which is called every time it is needed. Every shadeop must have a method. A method takes three arguments: a void *, which is the pointer returned by your init function (or NULL if there was no init function); an int indicating the number of arguments to the shadeop (including one for the place to store the return value); and a void ** pointing to an array of void *'s, each of which points to an argument. A method returns an int -- a return value of 0 indicates no error, a return value of 1 indicates an error.

The argv elements point to the arguments to the shadeop, starting with the return value, if any. In other words, for a non-void shadeop, argv[0] points to the place to store the result. For a shadeop that does not return a value, argv[0] is undefined. In either case, argv[1..argc-1] point to the arguments to the shadeop.

For arguments of type float, point/normal/vector/color, or matrix, the argument pointer is the address of a float or array of floats which are the actual values being passed. For string arguments, the argument pointer is the address of a structure of type STRING_DESC, which is defined in the header file shadeop.h. For arrays of any type, the pointer is directed toward a contiguous array of values of that type.

The return value, and also any changes to output parameters, may be written directly into the memory pointed to by the argument pointers, if it is any of the numeric types. However, you cannot simply write into the data for strings. The only way to write a string is to allocate the storage for the characters yourself, and assign its address to the char *s field of the STRING_DESC structure that the argument points to.

The optional shutdown function is called after all rendering is complete, and may be used to free allocated memory or otherwise clean up after the work done by the init and method functions. The shutdown function takes a single void * argument which points to the same data returned by the init function and passed to the method. The shutdown function has no return value.

The shadeop table

The C file contains a table describing the functions contained in the DSO -- essentially mapping the SL functions (with arguments) to the C implementations. Your functions may be polymorphic; in other words, you may have separate functions of the same name, which are distinguished by the types of the arguments passed to them, or by the type of the value returned by the function.

The table is an array of type SHADEOP_STRUCT, defined in shadeop.h. Declaration of the struct is eased by a macro, SHADEOP_TABLE. The struct has three members, all of type char *, which represent the declaration of the SL function, the name of the init function, and the name of the cleanup function. The end of the table is denoted by a SHADEOP_STRUCT with NULL as its entries. A shadeop without init or cleanup function can denote its absence by supplying an empty string (i.e. "") as the name of the missing optional function.

To give a more concrete example, here is an example table which describes implementations of a squaring function, which can take several types of arguments:

SHADEOP_TABLE(sqr) =
{
	{ "float sqr_f (float)", "", "" },
	{ "point sqr_triple (point)", "", "" },
	{ "vector sqr_triple (vector)", "", "" },
	{ "normal sqr_triple (normal)", "", "" },
	{ "color sqr_triple (color)", "", "" },
	{ "" }
};

Note the following properties of the table declaration:

1. The argument to the SHADEOP_TABLE is the name of the shadeop, as it will be called from a shader.
2. The table contains one entry for each polymorphic version of the new builtin function. In this example, there are versions which square floats, points, vectors, normals, and colors.
3. The table is a list of structs, each of which contains three strings. The end of the table is denoted by an entry which has an empty string as its first member.
4. The first string in each struct is a declaration of the SL function, giving argument types and order and return type, much like a C function prototype. The apparent name of the function in the declaration is the name of the actual C function used for the method.
5. The second string in each struct (not used in this example) is the name of the init function for the shadeop. (Note: it is the name of the function as a string, not the address of the function code.)
6. The third string in each struct (also not used in this example) is the name of the cleanup function for the shadeop.

Compilation Issues

Compiling and using a new DSO shadeop requires three steps:

1. Compiling the C file that contains your shadeop functions.
2. Compiling the shader that uses your functions.
3. Rendering a frame.

Compiling your C file is straightforward, just use the standard C or C++ compiler to generate an object (.o) file, then generate a shared object (.so) file from the resulting object file. If you use C++, you must use C style linkage. You must ensure that your build flags are compatible with the flags used to build the renderer.

SGI:   cc -n32 -mips3 -I/usr/local/prman/include -c myfunc.c
       ld -n32 -mips3 -shared myfunc.o -o myfunc.so
SUN:   cc -I/usr/local/prman/include -c myfunc.c
       ld -G myfunc.o -o myfunc.so
Alpha: cc -I/usr/local/prman/include -c myfunc.c
       ld -shared myfunc.o -o myfunc.so
Linux: cc -I/usr/local/prman/include -c myfunc.c
       ld -shared myfunc.o -o myfunc.so

The resulting file myfunc.so is the DSO that implements your new shadeop. It is not important that the filename matches the name of the shadeop.

When compiling your shader, if the SL compiler comes across a function reference which is neither a known builtin nor a function defined in SL, it will search all DSOs in the include path until it finds one (of any name) with the shadeop table for your function. Note that the .so file must be in one of the directories that the compiler searches for header files -- i.e. the path to DSO's must be specified with the -I command line option. The SL compiler will typecheck your function call and issue a warning if you pass arguments which do not match any of the entries in the SHADEOP_TABLE of your DSO.

Below is a simple example for implementing a squaring function, sqr(). It is polymorphic, in other words, you can pass either a float, a point-like type, or a color, and it will return a like type.

Please note that such a simple function is best implemented in SL itself, as the overhead of calling a DSO is greater than doing the multiply in SL.

sqr.c:

#include <stdio.h>
#include "shadeop.h"

SHADEOP_TABLE(sqr) =
{
	{ "float sqr_f (float)", "", "" },
	{ "point sqr_triple (point)", "", "" },
	{ "vector sqr_triple (vector)", "", "" },
	{ "normal sqr_triple (normal)", "", "" },
	{ "color sqr_triple (color)", "", "" },
	{ "" }
};

SHADEOP (sqr_f)
{
	float *result = (float *)argv[0];
	float *x = (float *)argv[1];
	*result = (*x) * (*x);
	return 0;
}

SHADEOP (sqr_triple)
{
	int i;
	float *result = (float *)argv[0];
	float *x = (float *)argv[1];
	for (i = 0; i < 3; ++i, ++result, ++x)
	{
		*result = (*x) * (*x);
	}
	return 0;
}

testsqr.sl:

surface testsqr (float angle = radians(30);)
{
	float x = 2;
	vector y = vector (1, 2, 3);
	printf ("x = %f, sqr(x) = %f\n", x, sqr(x));
	printf ("y = %p, sqr(y) = %p\n", y, sqr(y));
}

Compiling:

SGI:   cc -n32 -mips3 -I/usr/local/prman/include -c sqr.c
       ld -n32 -mips3 -shared sqr.o -o sqr.so
SUN:   cc -I/usr/local/prman/include -c sqr.c
       ld -G sqr.o -o sqr.so
Alpha: cc -I/usr/local/prman/include -c sqr.c
       ld -shared sqr.o -o sqr.so
Linux: cc -I/usr/local/prman/include -c sqr.c
       ld -shared sqr.o -o sqr.so