Frequently Asked Questions
From ShWiki
This document presents several Frequently Asked Questions (FAQs) and their answers about the Sh GPU Metaprogramming Language. If you have a question about Sh, look here first to see if it's already answered.
If you have a comment about this FAQ, please send an e-mail to the mailing list at [email protected] or click on the "Edit" link at the top of this wiki page.
Contents |
What is Sh?
See Sh for a quick introduction and How Sh Works for more details about the reasoning behind Sh.
How is Sh capitalized? How is it pronounced?
Like so: Sh. With a capital S and a lowercase h.
We usually pronounce the two letters in English. Although you're free of course free to pronounce it in any other language. But we don't recommend trying to pronouncing it as an actual word, as people will think you're telling them to quiet down.
Why did you choose this name? How does Sh relate to the Unix shell "sh"?
Sh originated from a larger research project called SMASH, which involved exploring not only software programmability of GPUs, but also hardware features. Sh was to form the software layer of the SMASH system. Eventually it was discovered that the name SMASH was already in use by a related project, so the two main parts were split into Sm (for the simulator) and Sh (for the language), conveniently making SMASH an acronym for "Sm and Sh".
While it is unfortunate that the name Sh clashes with the name of the Unix shell "sh", we decided after some deliberation not to change the name, as it was already in use publically for some time. Furthermore, the UNIX shell sh has largely been replaced by other derivatives such as bash and csh, and the contexts in which one speaks of (UNIX) sh tend to be very different from those in which Sh (our language) is mentioned. That is, it is highly unlikely that GPUs or similar architectures will be targeted by the UNIX shell in the future.
What is metaprogramming?
Metaprograms are program that write other programs. Sh supports metaprogramming implicitly. You don't even have to realise that you're doing it, but when you write an Sh program, you're really writing a C++ program that generates an Sh program. If this seems odd, don't worry -- you don't really need to know about this to use Sh.
Comparisons with other languages
There are lots of other shading languages out there. Cg, HLSL, and the OpenGL Shading Language are probably the most prominent ones in use on GPUs today. There are also of course "offline" shading languages like Renderman, but we will not treat those here.
While all of the other GPU shading are essentially syntactic variants of one another, with at most a few extra function calls, Sh is something entirely different. With all of the other shading languages, shaders are written either as strings in the host application, or as separate files. With Sh, shaders are written directly in C++, together with the host application. This makes a significant difference. Shaders can share variables with the application -- eliminating the need for parameter binding (glue) code. For small shaders this more or less means throwing away half the code necessary to write a shader! The host application can very easily generate GPU programs on the fly, without having to generate some sort of text representation of the programs. This is very important for generating subtle variants of shader programs, e.g. for different numbers of light sources, or varying levels of complexity. Sh also integrates texture handling, taking away much of the burden associated with updating textures and using them from shader programs.
Apart from this crucial basic difference between Sh and other shader languages, there are a few other notable differences that deserve mentioning. For one, while all of these shader languages are free, Sh is completely open source. Some other language provide an open source frontend (the lexer and parser) but Sh makes everything freely available, including the optimizer and code generators. The Sh licence is very permissible, stating that you can basically do anything with the language except claim to have written it. This means you can integrate Sh into commercial applications without being under any sort of obligation, and can make your own derivatives of it.
Sh is also cross-platform. While we currently only support Windows and Linux in the official version, we have a Mac OS X port that is to be integrated soon (at the time of writing).
Being mostly written by university researchers, Sh has no strong ties to a particular platform, API (such as OpenGL or DirectX) or graphics vendor. Compared to Cg, HLSL and the OpenGL Shading Language, it is the most independent shading language out there.
The research origins of Sh also mean that we are very interested in trying out new ideas in relation to shading. For example, our Shader Algebra paper from SIGGRAPH 2004 illustrates a mechanism built into Sh to take existing Sh programs and easily generate modified programs from these at runtime.
Lastly, our goal for Sh is to not just be a shading language. GPUs are becoming increasingly suitable for General Purpose computation, and this is interesting for a number of reasons, not the least of which is the amazing speed that GPUs offer at commodity prices. Thus we are aiming for Sh to be a language suitable for any kind of computations on GPUs, whether graphical or not.
Is Sh slower than Cg/HLSL/OpenGL shading language?
The short answer is: No, there is nothing intrinsic about Sh that makes it slower than Cg, HLSL or the OpenGL shading language.
As always, there is a longer answer too. Sh is really a GPU programming toolkit - it does much more than other shading languages. For example, Sh can set up textures for you, handle your uniform variables automatically (no more glue code!) and allow programs to be transformed in various ways. Some of these features may be more efficiently implemented manually. For this reason, Sh also tries very hard to give you, the developer, the flexibility to set things up on your own. For example, textures and uniform variables can be bound to existing OpenGL state in the OpenGL backends, allowing you to set these up as you see fit. This effectively makes Sh behave the same way other shading languages do, allowing more flexibility but taking away a lot of the convenience it provides by default.
Furthermore, Sh includes a full compiler. This includes an optimizer, register allocator and code generator. It is certainly the case that for certain programs other shader compilers will generate better code than Sh. However, the converse is also true - there are optimizations which at the time of writing only Sh performs (e.g. automatically determining subexpressions that only depend on uniforms, and lifting these to the host instead of the GPU). However, we are also working on adding compiler backends for Sh that target other high-level languages (for example, at the time of writing we are very close to completing our OpenGL Shading Language backend). This will allow you to take advantage of vendor-supplied compiler optimizations if you find that Sh's optimizer is not sufficient.
How does Sh compare to Brook?
Brook is a stream processing language developed at Stanford. Brook for GPUs is an implementation of Brook that runs directly on modern GPUs. This is implemented as a translator frontend that generates Cg code together with a C++ runtime.
Sh also has a stream processing subset. The main functional difference between Sh and Brook is that Brook is intended only for general purpose computation, whereas Sh targets both graphics and general stream processing. Brook has features that Sh lacks, and Sh has features not found in Brook, but both projects are under active development.
How does Sh work?
This is a summary of a more detailed explanation of how Sh works.
Sh is simply a C++ library. Matrix-vector libraries, for small-scale matrix and vector operations, are commonplace amongst graphics programs. To the outside Sh looks (and acts) like such a library. You can add vectors together, multiply matrices with vectors, perform dot products, etc. Sh provides a number of types to represent this kind of data and uses operator overloading and other standard C++ features to make using these types convenient.
What separates Sh from other matrix-vector libraries is that every operation has two implementations. The default implementation runs in what we call immediate mode. In this mode, when you ask for two numbers to be added, the computation is performed and the result returned at that time.
The second mode is retained mode. When you turn on a special flag, at runtime, all the operations in Sh switch from performing a computation to recording a computation. This means that all the operations you specify in a particular instance of retained mode are collected together into a sequence of operations forming a program. Implementing this is not difficult (for example, there is no special template magic involved) but has deep consequences. When you use Sh in retained mode it looks like you are writing operations in C++, but those operations are really compiled at runtime into a program by the compiler portion of Sh. Sh includes a full optimizing compiler that targets several GPU and CPU backends.
The true power of using Sh comes into play when immediate mode and retained mode are mixed. Variables with Sh types declared inside a retained mode program belong to that program. Variables declared outside (i.e. in immediate mode) belong to the host application. If you use a host variable from an Sh program, it becomes a uniform parameter of that program. In other shading languages you would have to explicitly manage the relationship between host variables and program variables, incurring lots of inconvenient glue code (which can sometimes be as long as the shaders you are writing). In Sh, updating a uniform variable becomes as easy as assigning to it in C++. In essence the C++ scope rules are being used to manage the relationships between host code and shader code. This powerful idea extends to all other forms of C++ abstraction, enabling you to completely use functions, classes, namespaces and other forms of C++ modularity when using Sh.
Does Sh use a preprocessor? Does Sh have a separate compiler?
No. Sh is plain ISO standard C++, and is provided simply as a set of header files to include and a shared library (DLL on Windows, .so on other platforms) to link with. No preprocessor or separate compiler is necessary.
Does Sh use template metaprogramming?
No. Template metaprogramming is a technique that allows arbitrary computations to be performed at C++ compile time using C++ templates. Sh does not need to use any such techniques. Sh code is compiled dynamically at runtime (similar to a just-in-time compiler). The mechanism for this is simple, and does not require any non-traditional use of templates. Metaprogramming in relation to Sh usually means the ability to have the host application define shaders at runtime (and therefore generate shaders programmatically).
Sh does make use of C++ templates, but only in the traditional sense, in order to make it more type safe while keeping the language general. Using templates for this makes the Sh code substantially shorter and more maintainable.
Getting Sh
You can always get the latest version of Sh, Shrike and Shmedia through our Downloads page.
See the COPYING file to read the licensing terms.
For examples, look in the examples directory or have a look at the Sample Code page.
Subversion Access
Subversion is the Version Control System (VCS) we use to store all of the Sh source code and related data. While it is similar to the more well-known CVS, it solves many of the problems that CVS has, and is much cleaner overall. We recommend anyone looking for a solid version control system to look into Subversion. On Windows, we suggest using TortoiseSVN.
The Sh subversion repository is located at
https://svn.libsh.org
For anonymous read-only access use the username "anonymous", with an empty password.
Using Sh
Is Sh usable? What state is Sh in?
Sh is usable as a real shader and stream language today, and is constantly under development. For a list of outstanding issues, please see our issue tracker.
What do I need to use Sh?
The current Sh backends for shaders target programmable floating-point GPUs. For fragment programs, this means NVIDIA GeForce FX 5200 and up or ATI Radeon 9600 and up (a variety of 3DLabs cards are also supported). For vertex programs, a variety of older hardware is supported (any hardware that supports ARB_vertex_program on OpenGL should work).
For stream programs, Sh also includes a CPU backend that generates C code and compiles and links it at runtime. This is useful for debugging, for performance comparisons or for testing features which are not available on the GPU yet. Usually switching between the GPU and the CPU is a single line change.
Sh is currently supported on Windows and Linux, and we are working on MacOS X support. We require at least the Visual C++ .NET (7) compiler on Windows, or the gcc compiler (version 3.3 and up) on Linux. Both of these compilers are freely available.
Can I use Sh with Microsoft Visual C++ 6?
Unfortunately no. Microsoft Visual C++ 6 has many severely broken language features. Sh is written with a standard C++ compiler in mind, and supporting Visual C++ 6 would have a significant impact on the maintainability, readability and potentially performance of the Sh code.
However, Sh support Microsoft Visual C++ .NET (also called Visual C++ 7 by some) and up out of the box. We recommend you upgrade to this compiler. Note that the command line version of this compiler is available for free from Microsoft, and detailed installation instructions for Sh using this and other compilers are available from our website.
Support
Is Sh commercially supported?
RapidMind Inc. is a company founded by the people behind the Sh project. If you have a commercial interest in Sh, seek a support contract, or wish to have a commercial extension developed, please contact [email protected] and we will be happy to assist you.
How can I help out?
We welcome all kinds of contributions to Sh. There is always much to do, so feel free to volunteer. If you are interested, you should start by looking through the list of open issues in our issue tracker (https://issues.libsh.org/) and by e-mailing the libsh-devel mailing list, letting us know that you're interested in contributing.
Where can I buy a copy of the Sh book, Metaprogramming GPUs with Sh?
See Metaprogramming GPUs with Sh.
I think I've found a bug! How do I report it?
Please use our issue tracker. Feel free to create an account for yourself. We also recommend joining the libsh-users mailing list and discussing it there.
Be sure to check the issue tracker first to make sure your issue has not already been reported.
Is there an IRC channel for Sh?
Yes! Simply point your IRC client at irc.freenode.net, and join the channel #libsh.
