Client/Server Protocol

This page documents the Wyvern Client/Server bi-directional network protocol.

Contents

• Protocol Overview
• Protocol Data Content
• Connecting to Server
• Initial Handshaking
• Client Writes
• Client-To-Server Protocol
  • User Commands
  • Mouse Input
  • Special Commands
    • The sel command
• Server-To-Client Protocol
  • Packet Structure
  • RPC Codes and Data
  • Full-Screen Updates
  • Partial-Screen Updates
  • Terrain Borders
  • Text Styles
  • Inventory Resend
  • Alpha Compositing

Protocol Overview

The Wyvern client/server protocol has the following basic characteristics:

• It's a low-level, TCP socket-based, custom RPC protocol.

• Data input and output occurs over the same socket connection.

• The client always sends text/UTF-8 data to the server.

• The server sends binary and UTF-8 data to the client, which the client must decode manually.

• Most binary data coming from the server is compressed using the gzip format. It's compressed on the server side using a java.util.zip.GZipOutputStream. You'll need access to a library that provides you the ability to uncompress (gunzip) the data stream on the client side.

• The connection is not secure, and doesn't use SSL or any form of encryption. Passwords are sent as text, encrypted with the crypt() command in the standard C library. You'll need to obtain access to a crypt() function for your platform, in order for your login information to be accepted by the server. You do not need to be a Wizard character to develop a Wyvern game client.

• The server writes int/short/byte values as high-byte first. So reading an int, you'd read the high byte first, and the low byte last.

• All strings are passed in UTF-8 encoding. The first 2 bytes is a short value representing the length of the string data.

Protocol Data Content

The information passed back and forth consists of:

• initial handshaking data to get the player logged in

• text commands and their arguments from the client

• RPC commands from the server, consisting of a one-byte code used to determine which method to call on the client, followed by a method-specific stream of bytes to decode.

  For example, the server might send the byte value INV_ADD, which means "add an item to the player's inventory view", followed by the item's name (utf-8), popup menu commands (utf-8), and image tile number, list index, and image (x,y) panning offset, the latter three of which are 16-bit unsigned short (int) values.

Because the protocol is TCP, there are no timeouts or retries built into the protocol. If the socket is closed, the client should disconnect and show the user the "connect to game" UI.

For the rest of this document, we're going to use the following naming conventions:

Any ALL_CAPS_WITH_UNDERSCORES is an RPC Function Code. It's a one-byte code that the server sends, to tell the client which function it's invoking.

An RPC Function Code is usually followed by some data, and this data will be called the message data for that RPC function. The data is a essentially a sequence of bytes, sometimes gzip-compressed, that needs to be decoded into one of (usually) these four types:

1. a single primitive-type int value (32-bit int, 16-bit short, or 8-bit byte). Sometimes these values are treated as signed, and sometimes as unsigned. The formal protocol definition shows how to decode and interpret each function's data stream.

2. a utf8 text string, where the first two bytes is the length of the string. The strings are not null terminated.

3. an array of bytes, shorts, or ints

4. a multi-dimensional array of bytes, shorts, or ints

5. a composite message consisting of any combination of the types above.

For instance, the message data for the INV_REMOVE function is simply a 16-bit short value saying which index to remove from the inventory view. The INV_MODIFY function has more data: 2 UTF-8 strings, followed by three 16-bit short values. The ZIP_SCREEN and ZIP_PARTIAL_SCREEN functions have fairly complex message data formats that require several stages of decoding and interpretation. They are all described in detail in the formal protocol definition, below.

For reference, the data is always encoded on the server end using a java.io.DataOutputStream writing to a java.io.ByteArrayOutputStream. You can consult Sun's Java 2 Platform APIs (JDK 1.4) if you'd like more information about these I/O classes.

Connecting to Server

The client should open a TCP socket to the host specified by the player, or "opal.cabochon.com" by default. Note: as of Jan 10 2003, opal is the only game server online. There is currently no way to run a local copy of the game server, so you'll have to do your testing by connecting to our production servers. We will try to offer a test server at some point.

The graphical-client port number is 2222. You should ensure that your local firewall lets traffic through on this port.

You can tell if the game server is up in 3 ways:

1. You can telnet to the game server, port 2000, using any standard telnet program. You don't have to have a character to do this, although it won't let you log in without one. However, if you connect with telnet (e.g. "telnet opal.cabochon.com 2000"), you should get a text-only welcome message and a login prompt from the server. If your connection is refused, the server is down (for maintenance or a scheduled reboot.)

2. You can visit this URL to see if there are people playing. If so, the server is up.

3. You can connect to the game using an existing graphical client (usually the Java Wyvern Client). If you don't have Java (JDK 1.4) on your platform, you'll be flying somewhat blind, so we recommend that you have a Windows or Linux PC available for running the standard client.

In order to test most of the functionality of your client, you'll need to have a character on Wyvern. You can create one (entirely free of charge) using our New User Signup Form. Your password will be emailed to you by the server.

Once you've established that the game server is available, you can start writing the code for the handshake.

Initial Handshaking

The handshake is a short exchange of data:

1. The client sends some registration data to the server:
   • an int whose value is 1 (REGISTER_EXISTING_PLAYER)

   • two short values: the view width and height being requested by the client. They should be positive, odd numbers, and the server will limit them to some preset value (currently 13).

   • a UTF string: the player's name.

   • a UTF string: the player's encrypted password

   • a UTF string: the client version. This is in one of two formats:
     1. client-type version-number (e.g. "Java WebStart Client 1.1")
     2. PDA handheld-name version-number (e.g. "PDA IPAQ PocketPC 2.2")

     The client type is mostly for logging purposes, so we can see what kinds of clients are connecting to the game. PDA clients get some special handling, as well - the player goes to a different starting area, and there are other minor differences in the gameplay. The protocol is the same for both.

   Note that the protocol sends and receives UTF8 strings as 2 bytes (the length of the data) followed by the UTF8-encoded data. Most UTF8 strings sent from the server are preceded by a 1-byte code specifying the text style to draw the text in. This part of the protocol will be changing in the near future.

2. The server responds with a packet:
   1. an int: the packet header. All packets from the server have an int packet header, described below.

   2. an int: the major and minor protocol version numbers that the server is using.

   3. a byte: LOGIN_SUCCESS (value=3), or LOGIN_FAILURE (value=4).
      • On LOGIN_SUCCESS, the handshake is finished, and it's the end of the packet.

      • On LOGIN_FAILURE, the packet includes a style byte and a UTF8 error message. The server will follow the packet with a QUIT packet, described below.

After that, the server will begin sending data in packets, but they won't necessarily come in any particular order.

After the handshake data is complete, the client should begin polling the socket for data from the server. The client must also respond to user input to send commands to the server, over the same socket. Reads and writes can be arbitrarily interleaved; the protocol is stateless after the handshake completes.

Client Writes

After the initial handshake is complete, the client can begin sending text commands to the server. The commands fall into three categories:

1. Input typed directly by the user, such as get all from corpse

2. Mouse clicks, which are translated into text strings like mouse 7 13 or inv 3 247. These are described below.

3. Special commands sent by the client to request specific server functions, such as asking for an image or sound clip to be dowloaded, or asking for a full-screen refresh.

All input from client-side UI objects is converted to commands, as if the player had typed the command at the command-line.

The server expects the client data to be UTF-8 encoded. The server first reads a 2-byte value that contains the length of the UTF8 data. Then it reads the data. There is no null terminator.

The gory details follow.

Client-To-Server Protocol

User Commands

Generally, you send exactly what the user types into the text-input field directly to the server as a utf8 string.

The client should limit the length of the input to approximately 2000 characters. If the server receives too much input from a client in one read, it will deliberately close the connection.

Mouse Input

When the user clicks the mouse in the map, the client should generate a text command of the form mouse x y modifiers, where:

• x is the viewport x "tile" coordinate where the user clicked. The client viewport should ideally be evenly divisible by 32 pixels, since the image ("tile") size for the game is 32x32 pixels. If the user clicks the mouse in the upper-left corner of the viewport, it would be location (0, 0), with x increasing to the right and y increasing downwards.

• y is the viewport y tile coordinate. You typically compute x and y by diving the pixel (x,y) locations of the mouse press by 32.

• modifiers is a numeric string whose int value is a bitfield, with bit representing the Control, Alt and/or Shift keys, and mouse buttons 1, 2 and 3.

  The flags that the server looks for are:

  	 shift key:   0000 0001 (0x00000001)  "1"
  	 ctrl key:    0000 0010 (0x00000002)  "2"
  	 alt key:     0000 1000 (0x00000008)  "8"
  	 button 1:    0001 0000 (0x00000010)  "16"
  	 button 2:    0000 1000 (0x00000008)  "8"
  	 button 3:    0000 0100 (0x00000004)  "4"

  Where, of course, the shift-key bit is set if the user was holding the Shift key down when the mouse press occurred, and so on. You should || (or) in each bit, convert to a decimal integer, and then to a string.

  For example, shift + left mouse button (button1) is 1 + 16, so you'd send 17 for the modifiers.

  The ramification of this numbering scheme, described in more detail below, is that there's no way to specify two combinations: ctrl+button3, and alt+button2.

See the sel command for how the mouse is handled when you click in the ground or inventory views.

Special Commands

The client needs to send a handful of special commands as the need arises:

• #tile - asks the server to send the specified tile mapping. Argument is the tile number for which we need a mapping. Example: #tile 2117

  The client should send this command when it gets a request to draw a tile that isn't in the client's tile-mappings hash (meaning server hasn't sent it yet.)

• #img - sends a request to the server to send us a particular image. Argument is the path to the image to download. Example: #img monsters/goblin/orc

  You send this command when you've received a tile number from the server for an image whose timestamp is older than the server timestamp for that image.

• #view - tells the server to change the size of the player's camera to a new width and height, in map coordinates (or viewport "tile" coordinates). Example: #view 7 5 is sent by the PocketPC client when the user wants to shrink the map from 7x7 and show more of the text-output window.

• refresh - instructs the server to send a full-screen refresh. There are no arguments - just send refresh. Use this when the client's view size changes, or when anything happens to cause the client to forget how to draw the current map contents.

• sel - sent when the user chooses a menu command for an item in the ground or inventory view. This command is described below.

• player-visible commands - the client can, at its option, offer UI shortcuts for player commands. For instance, the default Java client has a row of icon buttons that send commands like who, score, and so on. The full list of player commands is available in the Player Command Reference.

The sel command

The client should send the sel command to the server when the user clicks in the ground or inventory view.

sel [gnd|inv] <index> [modifiers|command]

Where:

• gnd or inv specifies which view was chosen

• index is the index of the item selected (0-indexed)

• modifiers is an int value consisting of flags specifying whether the ctrl, alt, and/or shift keys were held down when the user clicked the mouse, as well as which mouse button it was. The flags recognized by the server are:
  • shift mask: 0x00000001 (1)
  • ctrl mask: 0x00000010 (2)
  • alt mask: 0x00001000 (8)
  • button1 mask (left): 0x00010000 (16)
  • button2 mask (middle): 0x00001000 (8)
  • button3 mask (right): 0x00000100 (4)

  Note that alt-mask is the same as button2-mask - this is for historical reasons (they were the same in older versions of Java). So holding the alt key down emulates the middle mouse button.

  The modifiers are logically OR'ed together, and the int value is converted to a decimal number as the argument. For instance, if the user isn't holding down any keys, and presses the left button, the modifiers argument is "16". If the user holds down the Shift and Alt keys, and presses the left mouse button, the modifiers argument is 16 + 8 + 1 = "25".

• command a one-word command verb, such as drop, loot, get, or apply. The verbs that are valid for a given item are sent from the server in the INV_ADD, INV_MODIFY, and INV_RESEND RPC calls.

The client should send either the event modifiers or a command, not both. It's up to the client to decide how to map the mouse buttons - it's a good idea to create UI that lets the user decide. For instance, in the default Java client, the right mouse button brings up a menu containing the command verbs, and the left button examines an item (configurable by the player using the mousebind command, since the binding is stored on the server side.)

An Apple Macintosh client, in contrast, might opt to bring up the command menu when pressing the single mouse button, and emulate the shift, ctrl and alt key codes when the user presses some appropriate key on the Mac keyboard.

The modifiers argument is designed rather poorly; it relies on the actual input event mask values specified by Java, and the values have changed from release to release. At some point we'll add an entirely new set of flags that unambiguously differentiate between the modifier keys and the mouse buttons. The current flags will continue to be recognized by the server, for compatibility with older clients.

Similarly, the mouse customization level available to players is currently somewhat restrictive. We will eventually add the ability to bind logical keys ("shift", "alt", "ctrl") and mouse buttons to actions in the game. There are valid arguments for doing this on the client or the server. Saving the mapping on the server side means it applies no matter which computer you're playing the game from; saving it on the client means your mappings are valid for all of your characters. We'll undoubtedly offer a server-side binding mechanism, and leave it up to the client implementors to decide whether to offer a client-side binding as well.

Server-To-Client Protocol

Packet Structure

Every packet sent by the server, including the handshake packet, starts with a 4-byte integer header:

• The high-order byte is an RPC code. The codes are listed in the table below.

• The remaining 3 bytes hold the length of the packet data, not counting the header. So for RPC codes with no data, the 24-bit length value will be zero.

The length is sent for convenience to client writers. It allows you to skip over packets that you don't know how to handle, and it also makes decoding the data more convenient.

RPC Codes and Data

The following table lists the RPC codes currently in use by the client/server protocol. Each code is the first byte you receive in a request from the server. It tells you what function to invoke on the client. The byte is usually followed by some data - the arguments to the request.

Full-Screen Updates

The data from a ZIP_SCREEN request is a bit complicated. The server sends two data structures:

• a 2D terrain array of ints, consisting of information for drawing the terrain and borders. The array is 2 tiles taller and wider than the map view, since the terrain just outside the view is used in the terrain-bordering algorithm.

• a 1D array of object data, consisting of pairs of 16-bit signed short values. The first short holds the (x,y) coordinate at which to draw the object. The second short holds the object's tile number.

  For any objects that have alpha transparency, there will be a third short value that holds the alpha level (0-100). You can tell if there's an alpha value for a tile if it has the high bit set (0x8000). This value is described further in the Alpha Compositing section.

The data structures are written to the stream as follows:

• short: terrain array width (2 tiles wider than map view)
• short: terrain array height (2 tiles taller than map view)
• int: compressed data length
• int: uncompressed data length
• byte[]: a compressed buffer holding the terrain and object arrays

        // byte[] unzipped = (the uncompressed data buffer)
        // width = terrain array width
        // height = terrain array height
	int[][] terrain = new int[width][height];
	int count = 0;
	for ( int i = 0; i < width; i++ ) {
	    for ( int j = 0; j < height; j++ ) {
		int b1 = unzipped[count++];
		int b2 = unzipped[count++];
		int b3 = unzipped[count++];
		int b4 = unzipped[count++];
		if ( b1 < 0 ) b1 += 256;
		if ( b2 < 0 ) b2 += 256;
		if ( b3 < 0 ) b3 += 256;
		if ( b4 < 0 ) b4 += 256;
		int t = (b4 << 24) | (b3 << 16) | (b2 << 8) | b1;
		terrain[i][j] = t;
	    }
	}

Next, read the object data. Again, the code might be the easiest way to see how to do it:

    /**
     * Unzips the incoming data an performs a drawTerrain()
     * followed by drawObjects().
     *
     * @param width the width of the terrain array
     * @param height the height of the terrain array
     * @param len the length of the unzipped byte array
     * @param data a gzipped array of bytes representing the
     * int[][] for the terrain and the short[] for the objects.
     *
     * @author stevey
     */
    public void drawZippedScreen ( int width, int height,
				   int len, byte[] data )
	throws IOException
    {
	byte[] unzipped = Strings.uncompress ( data );

	// unpack the data
	int[][] terrain = new int[width][height];
	int count = 0;
	for ( int i = 0; i < width; i++ ) {
	    for ( int j = 0; j < height; j++ ) {
		int b1 = unzipped[count++];
		int b2 = unzipped[count++];
		int b3 = unzipped[count++];
		int b4 = unzipped[count++];
		if ( b1 < 0 ) b1 += 256;
		if ( b2 < 0 ) b2 += 256;
		if ( b3 < 0 ) b3 += 256;
		if ( b4 < 0 ) b4 += 256;
		int t = (b4 << 24) | (b3 << 16) | (b2 << 8) | b1;
		terrain[i][j] = t;
	    }
	}

	// unpack the remaining bytes into a short array
	int size = 0;
	int num = (len - count)/2;
	short[] objects = new short [ num ];
	while ( count < len )
	{
  	    int lo = unzipped[count++];
  	    int hi = unzipped[count++];

  	    if ( hi < 0 ) hi += 256;	// convert back to unsigned
  	    if ( lo < 0 ) lo += 256;

  	    objects[size++] = (short) ((hi << 8) | lo);
	}

	// draw the terrain, then the objects
	display_.drawAllTerrain ( terrain );
	display_.drawObjects ( objects );
	display_.flushGraphics();

    } // end of drawZippedScreen

    /**
     * Draws all the terrain in a specified rectangle.
     *
     * @param terrain an array of int-sized records containing enough
     * information to draw the terrain and borders.  The array is larger
     * than the view by 2 in each dimension, and the view is centered
     * in the array.  (That is, there's a 1-tile ring all the way around
     * the view of tiles you can't actually see in the view; they're
     * used to compute the borders for terrain at the view edges).
     *
     * @see wyvern.common.util.RemoteMap#drawAllTerrain()
     * @author stevey
     */
    public void drawAllTerrain ( int[][] terrain )
    {
	int width = terrain.length;
	if ( width == 0 ) return;
	int height = terrain[0].length;
	if ( height == 0 ) return;

	// draw the terrain in first.
	for ( int j = 1; j < height-1; j++ ) {
	    for ( int i = 1; i < width-1; i++ ) {

		// mask out high word
		int tile = terrain[i][j] & 0xFFFF;
		drawTile ( i-1, j-1, tile, 0 );
	    }
	}

	// draw borders in visible portion of the view
	if ( drawBorders_ ) {
	    drawBorders ( terrain, 0, 0, 0, 0, tileWidth_, tileHeight_ );
	}

    } // end of drawAllTerrain

    /**
     * Draws a sparse layer into the map.
     *
     * @param layer an array of alternating "stuffed" (x,y) view locations
     * and tile numbers for those locations.  If the high bit is set
     * on the tile number, the following short is extended info for
     * that tile.
     * @author stevey
     */
    public void drawObjects ( short[] layer )
    {
	if ( layer == null ) return;

	int i = 0;
	while ( i < layer.length )
	{
	    // extract the high & low bytes into their own word values.
	    // these are signed byte values from -128 to 127, so
	    // we have to specifically treat them as unsigned.  The
	    // ">>" operator preserves the sign for X, so we only
	    // need to do the subtraction for y.  (Why are negative
	    // values allowed?  Because the object's upper-left corner
	    // might be to the left of the view, or above it).

	    short temp = layer [i++];
	    int y = (temp & 0xff);
	    if ( y > 127 ) y -= 256;
	    int x = (temp >> 8);

	    short tile = layer[i++];
	    short alpha = 0;

	    // check 16th bit set
	    if ( (tile & 0x8000) != 0 ) {
		alpha = layer[i++];
		tile &= 0x7FFF; // clear it to get real tile number
	    }

	    drawTile ( x, y, tile, alpha );
	}

    } // end of drawObjects

Partial-Screen Updates

A partial-screen update logically consists of a list of squares to redraw in the map view. The squares don't have to be adjacent to each other: you could have, for example, the upper-left and lower-right squares redrawn as part of one partial-screen update.

The data comes as a compressed byte array. When uncompressed, it contains 3 sub-arrays. Each sub-array starts with an int for the length, followed by the data.

The three sub-arrays are:

1. squares to redraw
2. terrain info for those squares
3. non-terrain objects for those squares

	    short temp = locs[i];
	    int y = ( temp & 0xff );
	    if ( y > 127 ) y -= 256;
	    int x = ( temp >> 8 );

The second sub-array is the terrain data. It's a 3-dimensional array. The top-level dimension is the same length as the locations array. Each element of the top-level array is a 2d int array of terrain records used to compute terrain borders. Each 2d int array is a 3x3 array of integer "terrain records", described below. The center record, at location [1][1] in the array, is the terrain at the location being redrawn. The 8 neighboring ints are the terrain records for the 8 squares bordering the location to redraw.

The third and final sub-array is another short array, just like the first one. The values are tile numbers of the objects to draw at each location specified in the first sub-array.

Terrain Borders

Each square gets one integer bitfield, called a "terrain record", that's laid out like so:

               +-------------------------------+
               |1|1|1| 13 bits  |   16 bits    |
               +-------------------------------+
               /         |            |
           border      layer         tile
            flags     priority       number

• The low 16 bits are the terrain tile number.

• Bits 17-28 are the terrain priority, a signed number. (Bit 28 is the sign bit).

• Bit 29 is the "has-borders" flag; true if the terrain has borders.

• Bit 30 is the "lets-in" flag: set if the terrain permits borders to encroach into it, clear if the square disallows incoming borders even from higher-priority squares.

• Bit 31 is the "lets-out" flag: set if the square wants to extend borders (only meaningful if the terrain type actually has borders), clear if the square doesn't want to use any borders it might normally have.

The code that actually implements the terrain-bordering algorithm is tricky enough that we've provided the java source code used in the game (in all the Java clients, and in the Map Editor). You can view it here:

It's certainly not the prettiest (or fastest) code in the world, but it works. It is one of the slowest pieces of the map-drawing code on the client, and we see significant speedups in the Java client when we turn off terrain bordering - an option available in one of the menus.

We obviously need to provide a great deal more information about the algorithm and the shapes of the 20 border tiles. We don't expect client writers to reverse-engineer it. Until this documentation is completed, however, you can draw your maps without terrain borders, as follows:

1. for each 3x3 terrain array, discard all but the center (1, 1) int value.

2. for the center value, mask out the high 16 bits, and the result is the tile number of the terrain to draw at that square.

Text Styles

The style codes are defined as follows:

Inventory Resend

The packet consists of a zipped byte array, which has the following standard format:

• an int containing the length of the zipped data to follow
• an int containing the length of the data when it's unzipped
• the zipped data

• an int: the number of bytes in the following string array
• an int: contains the number of strings in the byte array
• that many utf8 strings (each one 2 length bytes followed by the utf8-encoded data, as usual)

• an int: contains the number of short values to read
• that many short values

The arrays are defined as follows:

1. the item names

2. the commands for the items. Each string is a whitespace-delimited list of command verbs valid for that item.

3. the tile numbers for the items

4. the (x, y) tile drawing offsets for the items. Most of the time, these are zero, since most items in the game are one "tile" (32x32 pixels).

Alpha Compositing

Alpha and transparency are complex topics, and it's beyond the scope of this document to cover them. If you're unfamiliar with them, there are many tutorials on them on the web.

Wyvern actually supports two forms of specifying transparency:

1. Per-pixel full transparency. This is built into the GIF (and PNG) image file format. When you create an indexed GIF or PNG image in an image editor (such as Photoshop), you can specify that one of the 255 index colors is the "transparent color". Whenever you use that color in the image, then it should use the background color instead when you draw the image to the screen.

   This feature is used extensively in Wyvern to "fake" a 3-dimensional view. If you drop a shield, the area in the 32x32 image around the shield is transparent, and you'll see the ground there (or whatever else was under the shield.)

2. Per-image alpha blending. The partial-screen and full-screen updates (as well as SEND_PICTURE) support sending an alpha value along with an image. The alpha value is described in this section.

If the value is from 1 to 99, it means that the image should be blended with the background using an Alpha Compositing algorithm. Most programming frameworks provide support for alpha compositing (also called alpha blending) in their graphics system.

In most systems, the alpha component ranges from 0.0 to 1.0, where 0.0 is fully transparent and 1.0 is fully opaque. In Wyvern, you simply divide the alpha value by 100.0 to get the floating-point value from 0.0 to 1.0. You should first check if the value is 0 or 100, and don't do alpha blending for those cases.

Note that the PNG image format (which we'll migrate to at some point) supports per-pixel alpha values, at 8 bits per pixel. This requires 32-bit-per-pixel images, which are significantly larger (up to 8 times larger) than Wyvern's indexed 256-color images. We're not sure if this will result in a reasonable experience for modem users, so the migration is on hold for now. However, most images will probably NOT need transparency, so we may be able to get away with only using 32-bit images when it's required.