Building XML-RPC Clients in C
XML-RPC is a useful protocol for building cross-platform, client-server applications. Often XML-RPC is demonstrated with high-level interpreted languages like Perl and Python. In this article, Eric Kidd's XML-RPC C library is used to build a simple, yet powerful debugging client. Special care is taken to bring programmers with rusty C-hacking skills up to speed.
XML-RPC is a wire protocol that describes an XML serialization format that clients and servers use to pass remote procedure calls to each other. There are two features that make this protocol worth knowing. The first is that the details of parsing the XML are hidden from the user. The second is that clients and servers don't need to be written in the same language. For more background information on XML-RPC, check out the Resources listed at the end of this article.
Many articles written about XML-RPC use Java, Perl, Python, or PHP to demonstrate building Web Services. However, real life programming requirements often mitigate against the programmer's first choice of implementation language, perhaps because access to a resource whose only API is a C library is required. In this case, building an XML-RPC server to this resource opens it up to any client that supports XML-RPC.
In high-level languages like Perl or Python, development can be rapid because the compile-run-edit test cycle is almost as fast as opening one's editor. In C, this isn't the case. Compiling a program can take a lot of time. But sometimes it's the only tool for the job. If your C skills are rusty, this article is for you. Because XML-RPC is a high-level protocol, it takes a good number of other C libraries to make the magic happen. Remember, even a simple client needs to be able to talk HTTP and parse XML, both of which are far beyond the built-in facilities of C. The first step is installing Eric Kidd's C library.
Eric Kidd's C/C++ interface to XML-RPC requires the W3 Consortium's
WWW library. I used version
0.9.9 of the XML-RPC library and version 5.3.2 of the W3C's libwww for
this article. Both packages compiled cleanly on my Red Hat 7.1 box
using the standard ./configure && make, but there was a
catch during the make install phase. The names of the
shared libraries had an spurious '.0' appended to them. Those already
familiar with the way shared libraries are found on a Linux box may
skip the next section.
Joe Johnston has also written an article on Binary Data to Go: Using XML-RPC to Serve Up Charts on the Fly, which looks at how XML-RPC and Web services can create simple charts for client programs.
Both Kidd's and the W3C's libraries install shared libraries. These
differ from the static libraries you might be more familiar with in that they
are loaded during a program's runtime rather than being built into
a monolithic static binary. In order for the system to find these shared
libraries, you need to make sure that /etc/ld.so.conf lists
the path to your shared libraries. In the case of the XML-RPC libraries
listed above, /usr/lib must be listed. For existing binaries,
you can check whether the system can locate the shared libraries by using the
ldd command:
$ ldd <executable_name>
In the case of the program presented here, the ldd output will look something like:
[jjohn@marian src]$ ldd xmlrpc_debug
libwwwxml.so.0 => /usr/lib/libwwwxml.so.0 (0x4002f000)
libxmltok.so.0 => /usr/lib/libxmltok.so.0 (0x40032000)
libxmlparse.so.0 => /usr/lib/libxmlparse.so.0 (0x40044000)
libwwwzip.so.0 => /usr/lib/libwwwzip.so.0 (0x4004b000)
libwwwinit.so.0 => /usr/lib/libwwwinit.so.0 (0x4004e000)
libwwwapp.so.0 => /usr/lib/libwwwapp.so.0 (0x40051000)
libmd5.so.0 => /usr/lib/libmd5.so.0 (0x40067000)
libwwwhtml.so.0 => /usr/lib/libwwwhtml.so.0 (0x4006a000)
libwwwtelnet.so.0 => /usr/lib/libwwwtelnet.so.0 (0x40076000)
...
If a library doesn't have a hex address next to it, the system
can't find it. This can be solved by making sure that path in
/etc/ld.so.conf is listed and then running
ldconfig as root. Run the ldd command again
to make sure all your shared libraries are visible.
As I mentioned above, the gotcha is that Kidd and W3C libraries
don't name the shared libraries correctly (they install with an extra
'.0' on the name). You will need to create symlinks with the kind of
names that ldconfig expects. I used the Perl script in Listing 1 to do this, but any scripting tool
will do.
Listing 1: Fixing library names
1 #!/usr/bin/perl --
2 # Look for these libs; ensure symlinks are right
3
4 use strict;
5 use constant LIB_DIR => '/usr/lib';
6
7 my @libs = qw(
8 libwwwxml libxmltok libxmlparse libwwwzip
9 libwwwinit libwwwapp libmd5 libwwwhtml libwwwtelnet
10 libwwwnews libwwwhttp libwwwmime libwwwgopher libwwwftp
11 libwwwfile libwwwdir libwwwcache libwwwstream libwwwmux
12 libwwwtrans libwwwcore libwwwutils libdl libz
13 libxmlrpc libxmlrpc_xmlparse libxmlrpc_xmltok
14 );
15 if( $> != 0 ){
16 warn "ERROR: Must be root\n";
17 }
18
19 chdir LIB_DIR || die "ERROR: cd: $!";
20
21 for my $lib (@libs){
22 print "Looking for $lib\n";
23 my @candidates = glob("$lib*0");
24 for(@candidates){
25 my ($link, $real);
26 if( -l $_ ){
27 $real = readlink($_);
28 $link = "(is a symlink to $real)";
29 }
30 print "\t$_ $link\n";
31
32 if( $real ){
33 my $new = substr($_, 0, length($_) - 2);
34 print "\tcreating a new symlink($new) to $real\n";
35 unless( symlink $real, $new ){
36 warn "WARN: symlink: $!\n";
37 }
38 }
39 }
40 }
|
Once you've squared this detail away, you're ready to start coding.
|
When learning a new XML-RPC implementation, the most important thing to learn is how it deals with the XML-RPC datatypes listed in Table 1.
|
Recall that XML-RPC libraries map language-specific datatypes to XML-RPC datatypes. It makes sense, then, to build a client that has to deal with all possible XML-RPC datatypes. This program is normally invoked like this:
$ ./xmlrpc_debug http://somewhere.com/RPC2 get_array
The first argument is a URL to an XML-RPC server. The second argument is the name of a remote procedure that doesn't require input. The code for this client begins in Listing 2. As is typical of most C programs, various header files are pulled in.
Listing 2: xmlrpc_debug.c, part 1
1 #include <stdio.h>
2 #include <stdlib.h>
3
4
5 #include <string.h>
6 #include <xmlrpc.h>
7 #include <xmlrpc_client.h>
8
|
Listing 3 defines some constants and function
prototypes. As a point of style, I try to make default values constants. This
makes them easy to change, since most #define statements are
listed at the beginning of code.
Listing 3: xmlrpc_debug.c, part 2
9 #define NAME "DEBUG XML-RPC C Client"
10 #define VERSION "0.1"
11 #define SERVER_URL "http://127.0.0.1:3080/RPC2"
12 #define DEFAULT_RPC "status"
13
14 void die_if_fault_occurred(xmlrpc_env*);
15 void print_values(xmlrpc_env*, xmlrpc_value*);
16 int get_int(xmlrpc_env*, xmlrpc_value*);
17 int get_boolean(xmlrpc_env*, xmlrpc_value*);
18 double get_double(xmlrpc_env*, xmlrpc_value*);
19 char* get_timestamp(xmlrpc_env*, xmlrpc_value*);
20 char* get_string(xmlrpc_env*, xmlrpc_value*);
21 char* get_base64(xmlrpc_env*, xmlrpc_value*);
22 void get_array(xmlrpc_env*, xmlrpc_value*);
23 void get_struct(xmlrpc_env*, xmlrpc_value*);
24
|
The main line begins in Listing 4. Line 26
declares an important variable that is used by nearly every XML-RPC
function. The xmlrpc_env variable env will
contain error messages and other transactional information. You don't
need to worry about manipulating it directly, though. It's a kind of
black box.
Visit xml.oreilly.com for a complete list of O'Reilly's books on XML.
Line 32 initializes some internal flags, and line 33 sets up the
xmlrpc_env variable. These lines are mostly a stock
piece; you won't need to change them much for your own code.
Listing 4: xmlrpc_debug.c, part 3
25 int main (int argc, char** argv){
26 xmlrpc_env env;
27 xmlrpc_value *result;
28 char* url;
29 char* rpc;
30
31 /* Start up our XML-RPC client library. */
32 xmlrpc_client_init(XMLRPC_CLIENT_NO_FLAGS,
NAME, VERSION);
33 xmlrpc_env_init(&env);
34
|
Listing 4 examines the command line for parameters. If none are detected, default values are used. Again, if you are used to scripting languages, you might be surprised by the rather lengthy measures needed to deal with strings in C.
Listing 5: xmlrpc_debug.c, part 4
35 /* figure out URL */
36 if(argc > 0){
37 url = argv[1];
38 }else{
39 if( (url = (char *) malloc(sizeof(SERVER_URL))) ){
40 strcpy(url, SERVER_URL);
41 }else{
42 printf("ERROR: Couldn't malloc\n");
43 exit(1);
44 }
45 }
46
47 if(argc > 1){
48 rpc = argv[2];
49 }else{
50 if( (rpc = (char *) malloc(sizeof(DEFAULT_RPC))) ){
51 strcpy(rpc, DEFAULT_RPC);
52 }else{
53 printf("ERROR: Couldn't malloc\n");
54 exit(1);
55 }
56 }
57
|
In Listing 5 after 60 lines of code, I'm
ready to make the remote procedure call. In your own code, you will be
dealing with xmlrpc_client_call since it is the way to
send your request to the server. This function's third argument is a
kind of format string that tells the library how to map C data into
XML-RPC data. Although the code in Listing 5
doesn't actually pass any data, here's a brief snippet that passes one
string to a remote procedure:
result = xmlrpc_client_call( &env,
url,
"some_func",
"(s)",
"my great argument"
)
Here's a snippet that creates a struct:
result = xmlrpc_client_call( &env,
url,
"some_func",
"({s:i,s:i})",
"my great argument", 1,
"my next argument", 2
)
See the "overview.txt" document that's included with Kidd's XML-RPC
library for more details. The meat of this program occurs on line 70
when print_values is invoked. Given the result of the
remote procedure call, it simply decodes and prints out the values it
received. I'll show the implementation of this function below.
Listing 6: xmlrpc_debug.c, part 5
58 printf("Calling %s\n", url);
59
60 /* Call our XML-RPC server. */
61 result = xmlrpc_client_call(&env,
62 url,
63 rpc,
64 "()"
65 );
66
67 die_if_fault_occurred(&env);
68
69 /* What did we get? */
70 print_values(&env, result);
71
|
|
Listing 7 hints at the reference counting
that Kidd's XML-RPC library does. He tries to shield the user from the
details of memory management, but you must tell the system when you
are done with xmlrpc_value data (see line 73). There are
also two convenient functions (lines 76, 77) for cleaning up other
memory that was allocated behind the scenes.
Listing 7: xmlrpc_debug.c, part 6
72 /* Dispose of our result value. */
73 xmlrpc_DECREF(result);
74
75 /* Shutdown our XML-RPC client library. */
76 xmlrpc_env_clean(&env);
77 xmlrpc_client_cleanup();
78
79 free(url);
80
81 return 0;
82 }
83
|
Listing 8 shows how the six simplest data
types are decoded. The tricky part is knowing which "format" string to
pass to xmlrpc_parse_value. You can use this code as a
guide, but also look at the aforementioned overview.txt for the gory
details. Notice that after calling this conversion function, any
errors that may have occurred will be reported by
die_if_fault_occurred (which is code ripped entirely from
Eric Kidd's examples).
Listing 8: xmlrpc_debug.c, part 7
84 /*
85 subs
86 */
87 void print_values( xmlrpc_env* env, xmlrpc_value* in ){
88
89 /* What did we get back? */
90 switch (xmlrpc_value_type(in)) {
91 case (XMLRPC_TYPE_INT):
92 printf("Got an integer: %d\n", get_int(env, in));
93 break;
94 case (XMLRPC_TYPE_BOOL):
95 printf("Got a boolean: %d\n", get_boolean(env, in));
96 break;
97 case (XMLRPC_TYPE_DOUBLE):
98 printf("Got a double: %g\n", get_double(env, in));
99 break;
100 case (XMLRPC_TYPE_DATETIME):
101 printf("Got an ISO8601 timestamp: %s\n",
get_timestamp(env, in));
102 break;
103 case (XMLRPC_TYPE_STRING):
104 printf("Got a string: %s\n", get_string(env, in));
105 break;
106 case (XMLRPC_TYPE_BASE64):
107 printf("Got a base64 string: %s\n",
get_base64(env, in));
108 break;
109 case (XMLRPC_TYPE_ARRAY):
110 printf("Got an array:\n");
111 get_array(env, in);
112 break;
113 case (XMLRPC_TYPE_STRUCT):
114 printf("Got a struct\n");
115 get_struct(env, in);
116 break;
117 case (XMLRPC_TYPE_C_PTR):
118 printf("Got a C pointer?!\n");
119 break;
120 case (XMLRPC_TYPE_DEAD):
121 printf("Got a 0xDEADr?!\n");
122 break;
123 default:
124 printf("UNKNOWN XML-RPC DATATYPE\n");
125 }
126 }
|
Listing 9 is a bit long, but it's a pretty
simple switch statement. The XML-RPC library defines several constants
with which the xmlrpc_value that was passed in can be
identified. Most of these types should be self-evident. In order to
map XML-RPC values to C values, we need to call
xmlrpc_parse_value. Examples of this will be seen in the
small helper functions that begin with "get_". The last two cases are
internal datatypes that won't normally occur, but are shown for the
sake of completeness.
Listing 9: xmlrpc_debug.c, part 8
127 int get_int(xmlrpc_env* env, xmlrpc_value* in){
128 int i;
129 xmlrpc_parse_value(env, in, "i", &i);
130 die_if_fault_occurred(env);
131 return(i);
132 }
133
134
135 int get_boolean(xmlrpc_env* env, xmlrpc_value* in){
136 int i;
137 xmlrpc_parse_value(env, in, "b", &i);
138 die_if_fault_occurred(env);
139 return(i);
140 }
141
142 double get_double(xmlrpc_env* env, xmlrpc_value* in){
143 double d;
144 xmlrpc_parse_value(env, in, "d", &d);
145 die_if_fault_occurred(env);
146 return(d);
147 }
148
149
150 char* get_timestamp(xmlrpc_env* env, xmlrpc_value* in){
151 char *s;
152
153 xmlrpc_parse_value(env, in, "8", &s);
154 die_if_fault_occurred(env);
155 return(s);
156 }
157
158 char* get_string(xmlrpc_env* env, xmlrpc_value* in){
159 char* s;
160
161 xmlrpc_parse_value(env, in, "s", &s);
162 die_if_fault_occurred(env);
163 return(s);
164 }
165
166 char* get_base64(xmlrpc_env* env, xmlrpc_value* in) {
167 char *s;
168
169 xmlrpc_parse_value(env, in, "6", &s);
170 die_if_fault_occurred(env);
171 return(s);
172 }
173
|
Listing 10 shows how to handle the
simplest aggregate type, the array. Naively calling
xmlrpc_parse_value won't work here because each element
can be a different data type. Fortunately, the size of the array can
be determined with a call to xmlrpc_array_size. Then,
it's a simple matter to iterate through the array, retrieving each
element with a call to xmlrpc_array_get. In a fine
display of code reuse, this element is passed to
print_values.
Listing 10: xmlrpc_debug.c, part 9
174 void get_array(xmlrpc_env* env, xmlrpc_value* in){
175 int i, size = 0;
176 xmlrpc_value *el;
177
178 size = xmlrpc_array_size(env, in);
179 die_if_fault_occurred(env);
180
181 for(i=0; i < size; i++){
182 el = xmlrpc_array_get_item( env, in, i);
183 die_if_fault_occurred(env);
184 print_values(env, el);
185 }
186 }
187
|
|
Listing 11 demonstrates how to deal with
structs. Much like arrays, you can iterate over key-value pairs in
structs. The function xmlrpc_struct_size returns the
number of key-value pairs available. A call to
xmlrpc_get_key_and_value will point the two passed-in
pointers to xmlrpc_value pointers at the n-th key-value
pair. Although keys are almost always strings, I err on the side of
caution here by asking print_values to display the key's
value.
Listing 11: xmlrpc_debug.c, part 10
188 void get_struct(xmlrpc_env* env, xmlrpc_value* in){
189 int i, size = 0;
190 xmlrpc_value *key, *value;
191
192 size = xmlrpc_struct_size(env, in);
193 die_if_fault_occurred(env);
194
195 for(i=0; i < size; i++){
196 xmlrpc_struct_get_key_and_value(env,
197 in,
198 i,
199 &key,
200 &value);
201 die_if_fault_occurred(env);
202
203 printf("\tkey: ");
204 print_values(env, key);
205 printf("\tvalue: ");
206 print_values(env, value);
207 }
208 }
209
|
The program concludes with Listing 12. This is a routine that conveniently catches errors. It was also ripped in whole from the example code in the XML-RPC C library.
Listing 12: xmlrpc_debug.c, part 11
210 void die_if_fault_occurred (xmlrpc_env *env){
211
212 if (env->fault_occurred) {
213 fprintf(stderr, "XML-RPC Fault: %s (%d)\n",
214 env->fault_string, env->fault_code);
215 exit(1);
216 }
217 }
|
|
• Binary Data to
Go: Using XML-RPC to Serve Up Charts on the Fly |
Once the source code is ready, you'll want to compile it. There are a lot of shared libraries that need to be linked into any XML-RPC client program. Eric Kidd's documentation suggests that you initialize your variables like this:
$ CLIENT_CFLAGS=`xmlrpc-c-config libwww-client --cflags` $ CLIENT_LIBS=`xmlrpc-c-config libwww-client --libs`
I have simply expanded these so that I can track down missing library errors more easily. Listing 13 shows the barebones Makefile used to compile this debug client. All warnings are turned on and it should compile without warnings under gcc.
1 CLIENT_CFLAGS=-I/usr/include
2 CLIENT_LIBS=-L/usr/lib -lxmlrpc_client \
3 -lwwwxml -lxmltok -lxmlparse -lwwwzip \
4 -lwwwinit -lwwwapp -lmd5 -lwwwhtml -lwwwtelnet \
5 -lwwwnews -lwwwhttp -lwwwmime -lwwwgopher -lwwwftp \
6 -lwwwfile -lwwwdir -lwwwcache -lwwwstream -lwwwmux \
7 -lwwwtrans -lwwwcore -lwwwutils -ldl -lz \
8 -lxmlrpc -lxmlrpc_xmlparse -lxmlrpc_xmltok -Wl,--rpath -Wl,/usr/lib
9
10 CFLAGS=-Wall -ansi -pedantic -g
11
12 debug:
13 gcc $(CFLAGS) $(CLIENT_CFLAGS) -o \
xmlrpc_debug xmlrpc_debug.c $(CLIENT_LIBS)
|
The test harness for the debug client is a simple Perl XML-RPC server. It returns sample data of various formats. Like all Frontier::Daemon programs, it runs as a single-threaded HTTP server, listening, in this case, to TCP port 3080. The remote procedures this server handles are given in the methods hash, which is passed to Frontier::Daemon during initialization. Because these functions are so simple, anonymous subroutines are used instead of creating named subroutines and passing references to them. Normally, the Frontier::RPC2 library is good about guessing how to encode Perl scalars, but base64 strings need manual intervention. Line 26 in Listing 14 below shows this in action.
1 #!/usr/bin/perl --
2 # just return sample data
3
4 use strict;
5 use Frontier::Daemon;
6 use MIME::Base64;
7
8 use constant PORT => 3080;
9 use constant SERVER_URL => 'http://localhost:'.PORT.'/RPC2';
10
11 print "Starting: ", SERVER_URL, "\n";
12 my $coder = Frontier::RPC2->new;
13 Frontier::Daemon->new(
14 methods => {
15 str => sub{ return "AB "x512 },
16 int => sub{ return 12345 },
17 double => sub{ return 1.234 },
18 array => sub{ return [(0..128)] },
19 struct => sub{ return {
20 key1 => 1,
21 key2 => "value2",
22 key3 => 4.56,
23 }
24 },
25 base64 => sub{ return
26 $coder->base64(
27 encode_base64('I love Lucy')
28 )
29 },
30
31 mixed => sub {
32 return [
33 { key1 => 1.0 },
34 { key2 => 2.0 },
35 { key3 => 'three'},
36 ];
37 },
38 },
39
40 LocalPort => PORT,
41 Reuse => 1,
42 );
|
This code should give the aspiring C XML-RPC programmer a good starting point for building a real application. It's hard to beat a C program for execution speed. A program similar to the debug client shown here, but written in Perl or Python, will run five to ten times slower. Of course, the Perl or Python script is five to ten times faster to write. You can save either CPU time or programmer time, but not both. Happy hacking.
Joe Johnston is an independent contractor and freelance writer. A graduate of the University of Massachusetts in Boston with a B.A. in computer science, he is a teacher, a Web designer, and an author of articles for the Perl Journal and other publications. Joe coauthored O'Reilly's Programming Web Services with XML_RPC. He can be emailed at jjohn@cs.umb.edu.
O'Reilly & Associates recently released (June 2001) Programming Web Services with XML-RPC.
Sample Chapter 3, Client-Server Communication: XML-RPC in Java, is available free online.
You can also look at the Table of Contents, the Index, and the Full Description of the book.
For more information, or to order the book, click here.