Optimize Your Packets!

[grimsk8ter11]

Originally posted by Verrigan

Difficulty Hard 5/5

Packet Lag.. I've spoken about this before.. I submitted a tutorial on how to decrease the lag caused by the server sending large packets.. Today, I am going to try to explain how you can optimize your packets. (Dave, and anyone else for that matter, you are welcome to add to this tutorial if you like, but I will be placing the final tutorial in this original thread.)

Let's get down to business. Packets are the most important part of any online application. If you didn't know this, consider that there has to be communication between the server and its clients before your application can be online. If you don't have communication between the two, you defeat the purpose of creating an online application. This communication can happen in several ways.. Here are a few:

[*]MySQL - Allow server and client both access to a MySQL database to communicate with each other.. Both will have to read the database periodically to check for new messages.. (Not a very secure way to do business)
[*]Message Files - On a LAN, you could have a file (or files) on a shared drive to store messages.. Both server and client will need to access the file periodically to check for new messages.. (Also, not very secure)
[*]Sockets - You can send messages via sockets either in TCP (checks the receipt of the sent messages) or UDP (Doesn't check the receipt of sent messages). Server/Client wait for messages to arrive, and processes them. (Can be unsecure, but there are ways to keep good security..)



Since this is a tutorial for MSE, we will discuss how to use TCP to send messages back and forth. Currently, both server and client send strings (text) back and forth as their packets. We split the string using a separator character, and then check the packet type by comparing the first string. (The text before the first separator character) That string can be any length, and the number of bytes it uses is the length of the string..

The Current Way

For this example, I'm going to use the "MAPDATA" (MD) packet. So, we have the MD packet, and what do we know about it?

Well... With a base MSE, the MD packet can be from 2,677 bytes and up. It actually has a maximum size, but we're not worried about that right now. The size of the MD packet varies with what tiles/NPCs/etc it has on it.

Okay.. Why?

Think about it.. If you send a tile as "1" it uses 1 byte. But you need to think about the SEP_CHAR. So that's 2 bytes. Okay.. What if that tile is 10? Now you're using 2 bytes to send a byte's worth of data.. Add the SEP_CHAR, and it becomes 3. Now consider tile #200.. (All the way up to 255) You send 4 bytes of data to show 1 byte's worth of data. If your number is above 255, then you're sending an integer's worth of data.. (2 bytes), but it's still represented by 4 bytes+ in a string.

As Dave mentioned, each character in a string is a byte, so a 10 character string is equal to 10 bytes. This is how the packet is sent across the internet. You can pick it up on the other end as a string or as a dynamic byte array. Dynamic byte arrays work differently than static byte arrays.. See my prior tutorial on this for a description of how to declare dynamic arrays.

However, now we get to our memory issues. See, VB doesn't store strings as 1 byte per character. So when you receive the packet as a string, you're (behind the scenes) converting that data to UniCode. What this means is.. You are using 2 bytes per character (in memory) for each string.. So a 10 character string is using 20 bytes of memory. (Confused yet?)

So.. that 2,677 byte string you received is using up 5,354 bytes.. (Double what it needs. ) Great.. Twice as much memory is being used than is necessary. And don't forget the extra 10 bytes used by variable-length strings.. (Yes, the MSDN library apparently lies about this data type's size.. It's actually almost twice the size it claims) Do you think that might be why it takes longer to compare strings in VB? (I don't know, but it looks like a start.)

Before I move on to the next section, I want to assure you all that UniCode has its place in VB. Strings are not bad. They are very useful. But, if you don't need to use UniCode, it is better to use a byte array.

A Better Way?
You be the judge. I am just here to provide you with the information you can use to optimize your packets. It's tough to make the changes, and very time consuming. (It took me around 2 weeks to finish just the server-side packets.. I will gladly make those changes available as a download, once I finish this tutorial, for those who would like an example of how I did it, if enough people ask, and Shannara does not mind.)

For the most part, VB takes longer to compare strings than it does to compare numbers. Why? One reason could be because there is more bytes of memory to compare.. I won't kid you.. I don't know every aspect of why VB takes longer to compare strings.. I just know it does. If someone wants to elaborate on this, please feel free.

So, since strings take longer to compare, it makes sense to compare numbers, right? Okay.. How do we do that?

This is where the byte array comes into play. Luckily Winsock is on our side here. When you receive a packet with winsock, you can choose to either receive that packet as a string, or as a byte array. If you choose to receive that packet as a string, VB automatically converts that string to unicode for you. If you receive it as a byte array, it requires no change.. (Ooo.. saving a little time already. Don't go crazy.. You're only saving about .001 milliseconds for a 10 character string.. Yes, that was a guestimate. )

Okay, great. So we can receive the data as a byte array. Why would I want to receive it as a byte array? Here's the short answer: You will save bandwidth (tons of it over time) using byte arrays.

Okay... I am going to make the long answer as simplified as I can.

Let's recap. Strings use twice as much memory (in bytes) as the number of characters in them. As Shannara pointed out in his reply, "1" has a Len() of 1, but a LenB() of 2. Packets are sent across the internet as a list of bytes, whether you want them to be or not. If you send a string "1" over winsock, it is transmitted as asc("1"), or the byte value 49. When you catch that packet on the other side, you can either grab it as a byte, or as a string. If you catch it as a string, it is automatically converted to unicode. That's just a fancy way of saying it adds another byte value of 0 to it. You might ask yourself why that is.. And that's fine. Go visit msdn.microsoft.com for an answer, cause I'm not here to explain unicode to ya.

Now, when you send data back and forth between a server and client, you need to consider a couple of things. The first thing you need to consider is that packets are not always received exactly as sent. Oh, the order is still the same, but it might be broken up, making your 1 packet into 2 or more packets, so we have to have some way of being able to tell when our packet has ended. Another thing you need to consider, especially in a multi-user server/client environment, is how much data is being passed back and forth. The less data sent, the better. More data = lag for both client and server.

How do you tell when a packet ends? If you use a string, you can end each packet with a special character, or series of characters. But then, you also need to have some way of separating multiple pieces of data in those packets.. So your string is likely to end up being larger than necessary. If you use a byte array to send your data, you can have the first few bytes of that byte array tell the receiver how many bytes (after those) to wait for before processing the packet. In my method, I use 4 bytes, which is the size of a long, which would allow for a packet size of over 4 billion bytes.. (A little over-kill.. You could easily use 2 bytes, which allows for a packet size of over 65,000 bytes.)

Now, onto the next consideration.. The size of the data. If you use a string, you know that you have to send your numbers in the string, and as described above, you would end up using more bytes to transfer values that actually take up less space, so the smallest way to send data would be to copy the information you'd like to send into a byte array.

So now we know how to tell when the packet ends, and how we should send the packet. For this example, I will use the USEITEM packet. The old way, we send the packet as:

Code:

"USEITEM" & SEP_CHAR & InvNum & SEP_CHAR & END_CHAR

This way uses 11-12 bytes to send the packet.

With my way, I create a byte array called Buffer, and add the necessary data to it. In this case, the USEITEM packet would be: 3, 0, 0, 0, 15, 0, 11 (Breaking this down, the first 4 bytes show a long value of 3. The next 2 bytes show a packet type of 15, an integer value, for USEITEM, and the inventory number, which is 11 in this case.) A total of 7 bytes.. Saving 4-5 bytes. Now, you could easily use an integer (2 bytes) to show the length of the packet, and just 1 byte to show the type of the packet, which would drop the size of the packet to 4, saving 8-9 bytes out of that 11 we used in the strings.

So, when the server receives the packet, it can then use CopyMemory to move the first 4 (or 2) bytes over into a long (or an integer). Then, it can copy the next 2 bytes (or 1) into an integer (or byte). And finally, copy the last byte into a byte variable, and handle that information as it used to.

Below is a link to my modified copy of MSE. This MSE has the following things done to it:

[*]The server-side packets have been changed to byte arrays.
[*]IOCP has been added to the server via the JetByte COMSocketServer class DLL. (This was the same code I used to setup the IOCP tutorial, but the WinSock control works on the same principle, so you do not have to have IOCP to convert your packets to byte arrays.)
[*]I have implemented a "call-by-address" method, and separated each packet into a separate packet handling function on the server-side. This gets rid of the if-thens in HandleData.
[*]I have added modBuffer.bas to handle all the buffer (byte array) manipulation.. (AddByteToBuffer(), AddIntegerToBuffer(), GetByteFromBuffer(), etc.) This file is used on both server and client-side.
[*]I have added modMessages.bas to enumerate all the server-side messages, and for future placement of the enumerated client-side messages. This file is used on both server and client-side.



***** YOU WILL NEED TO REGISTER THE COMSOCKETSERVER.DLL FILE TO BE ABLE TO USE THIS EXAMPLE! Do this by going to Start/Run, and typing: regsvr32 \COMSocketServer.dll

Get your copy of my modified MSE here.

[edit]
Forgot to mention that I changed the packet length to an integer, and the packet type to a byte in the download.

[pingu]

Where is the download link?

[grimsk8ter11]

doesnt exist anymore. you, so youll have to know what your doing in some way from reading this.

[Misunderstood]

you don't have a copy of it grim? someone has to, your talkin bout the edited MS right?

[grimsk8ter11]

My copy is top secret xD, get me on MSN if you really need it though.

[Obsidian]

i have an unedited copy of this MSE if you want it, mis.

i'm not sure what grim is talking about... probably the same thing but the client has the packets completed as wel...

[Misunderstood]

Oh no, I don't want it. I think pingu does though.

[Misunderstood]

Hes alive :lol:

[Misunderstood]

Haha, perfect timing too. We're working on KoC again.

[Misunderstood]

ah thats nice, using your skills a little more .

[Minatours]

Cant argue wth money!

[phantasy]

howcomei cant compile the server, as in i can but the compiled version doesnt work?

: i have XP
: and i dont know any problems at all

[Spodi]

My only criticism is that, although you are creating a byte array to send, you are still sending in the form of a variant. This really isn't avoidable through the Winsock controls. I'd reccomend switching to the Winsock API so you can change that variant to a byte array and speed things up even more-so. You can also disable Nagling while you're at it this way, too, which will give you at least 100ms less ping. :wink:

[Gilgamesch]

is there an easy way to add this :lol:

i looked at the code and didnt really understand it :/

[Spodi]

is there an easy way to add this :lol:

i looked at the code and didnt really understand it :/

Simply - no. 8)

If you want to use binary packets, you're going to have to redesign every single packet once to fit the system, then again to optimize them.

Oh also, something I noticed is that you guys pack an integer to show how large the string is - you should rarely be sending strings over 255 characters long, so a byte should suffice. I created two functions - Put_String and Put_StringEX, where Put_String packs a byte and Put_StringEX packs an integer, and I only have to use Put_StringEX for when I am retrieving/writing the main body message of in-game mail. May not be that big of a change, but that is -1 byte for every string you send. 8)

[Spodi]

You pass it to the Winsock control as a byte array, but the Winsock control recieves it as a variant. This is why you can pass a string and a byte array through the same call.

From the object browser:

Sub SendData(data)
Member of MSWinsockLib.Winsock
Send data to remote computer

Remember what happens when you dont specify a data type in Visual Basic? That's right, it's a variant! 8)

[William]

howcomei cant compile the server, as in i can but the compiled version doesnt work?

: i have XP
: and i dont know any problems at all

Same problem here. Working in IDE (your server), but when compile, it crashes the hell out of it. .. on Win XP and Server 2003.

And yes, it's registered ...

Remember guys, change your IDE settings to the following:

menu > tools > options > general > error trapping, and enable "Break on all errors"

It will provide tons of errors (mainly type mismatches), that need to be fixed for a fully working client/server

Traced the bug down to the following code

Code:

Sub ClearTempTile()
  Dim tTempTileRec As TempTileRec 'Need object to obtain memory size.
  Dim tLen As Long

  tLen = (1 + UBound(TempTile) - LBound(TempTile)) * LenB(tTempTileRec)
  Call FillMemory(ByVal TempTile(1), tLen, 0)
End Sub

Replace with the following fixes the problem in that code .. but there's another ...

Code:

Sub ClearTempTile()
Dim i As Long, y As Long, x As Long

    For i = 1 To MAX_MAPS
        TempTile(i).DoorTimer = 0

        For y = 0 To MAX_MAPY
            For x = 0 To MAX_MAPX
                TempTile(i).DoorOpen(x, y) = NO
            Next x
        Next y
    Next i
End Sub

It looks like anything that uses FillMemory breaks when its compiled. It leaves alot to be fixed. I'll see if I can post a fixed client/server once I go through all the errors.

[Matt]

Maybe, you can also post a tut for fixing this too. That way, all the people who have already done extensive work on their game, can just add the fix, instead of having to search through the new client/server to find the fixes.

[pingu]

This artical explains the details of the different data types for VB .Net.. But I think some of it applies to VB 6.0, as well, which could explain why the Fill/ZeroMemory functions don't work well with arrays.. (Especially multi-dimensional ones)

http://msdn2.microsoft.com/en-us/library/47zceaw7.aspx

It's not exactly the same for VB6.

Thank Fal for this:

[pingu]

Heh, I just bothered reading the title of this topic an realized I had something to contribute...

I happen to be redoing the packet system using Spodi's system, which is just putting data types into a buffer and taking them out in the same order on the other side. I've had to redo everything involved with packets (sending and receiving) and it's taken me quite some time to do that.

The painful twist is that I'm converting Elysium and it's taken me almost three weeks to change the sending of the server and the recieving of the client (aka 50% completion). It hurts, but I've already noticed a difference when I managed to connect to the server one day. Elysium is known for the long "Recieving game data..." screens as it gets all the data from the client, but I swear it took less than a second with these changes.

Even then, I'm not even close to being finished (I do plan to release the source some time in the future). I'm getting a mysterious bug with this new system that shouldn't happen (of course it shouldn't happen, it's a bug...). I lose focus on the game screen and can't see anything (it's 100% black). I can still type messages and watch as the packets are sent from client to server and back again, but I can't move or even see the map. I'll figure it out soon enough...

Bleh, I just need somebody to feel my pain. If you can feel my pain, say "*feels pingu's pain*" and I'll give you a redeemable coupon for pingu feeling your pain.

[Vans]

*feels pingu's pain and its painful >_

[grimsk8ter11]

Well, i really cant explain anything amazing about this, because verrigan did it for us in KoC, so it was pretty much perfect, and i learned from looking at that sinc ei knew KoC before he added it like the back of my hand, and more or less, about 150 conversations about it with him because I had trouble using it in a base MS when i was working on MSE2.

Really, this is a godly to use if your starting from scratch with netwroking, which is what i did when i made the groundwork for the Moria engine. It makes things alot easier for you. The biggest problem with this in MSE is that MSE uses alot bigger Datatypes then it really needs to for SOOO many things that its not even funny, and over the course of changing them (especially if you tried to copy and paste from verrs example code, which did things mighty different then the best way to do them) you screw alot up. The hardest part was matching up datatypes for adding and removing to/from the buffer.

To Shan:
If you are having problem with the KoC code, talk to me please, because nothing should be wrong with it, since Verr did it (not saying hes perfect but he did quite a bit of testing if not more to make sure it worked, trust me, i know).

To Dave:
I had the same problem at first, the buffer seemed to like to overread it client side, check your incomingdata and ahndle data client side. If you want any help, ill be glad to offer assistance.

To Pingu:
Spodi's system IS verrigans system to tell you the truth, what you said PERFECTLY describes what verrigans system does, and there was some controversy over it for a while, untilt hey both decided to drop the argueing, or just got bored of it haha. But I feel your pain, I have done it countless times now, and each time, I get new problems, just takes alot of debuging. And if you need anby help, feel free to ask me too.

To Vans:
loser

To Verr:
Got to talk o you on Gtalk sometime, have some questions about Linely and such, also something about KoC.

To Mis:
You didnt post here, and i couldnt forget you just because im bored, but you're still a loser anyway. mwhahahaha :twisted:

[Spodi]

Spodi's system IS verrigans system to tell you the truth, what you said PERFECTLY describes what verrigans system does

Or the system used for many, many, many online games. I didn't even know about Verrigan doing this until a few months after I made it and wrote that guide, and I did it different then he did. It's a pretty common system to use, though. I didn't know there was "controversy" over it... Verrigan got annoyed with me for a completely different reason. :wink:

But I think some of it applies to VB 6.0, as well, which could explain why the Fill/ZeroMemory functions don't work well with arrays..

ZeroMemory/FillMemory works just fine with single-dimension arrays and multi-dimensional arrays... I've never had any problem with it.

[Misunderstood]

To Mis:
You didnt post here, and i couldnt forget you just because im bored, but you're still a loser anyway. mwhahahaha :twisted: l

You're awesome too.

[Dragoons Master]

Sry to bring this old topic back alive but I'm realy having some problems with the binary packets... I just downloaded the zip file verrigan gave us the link(http://www.verrigan.net/downloads/MSE-Verrigan.zip) and, well, I'm having problems with that! Can anybody download it and just run the server, run the client, create a char, login, and try to drop a gold coin(edit the account.ini). I do this and it just bugs for me... it log me off. I know is something with long values and only with them becouse I added it to my game and it works perfectly but not with any long values. Actualy it works once but when you send a second long values(doesn't mather when) it will bug... I don't know if it's a problem with me but I tried to redownload and it just keep happening... If anybody can help me out with this I would realy apreciate. Tnx in advance and once again, sry for realiving the topic.

EDIT: Omg, I can't beleave it... I found the bug... It was the last thing I would ever thing of... the COMSocketServer.dll DLL changed, I tought it did but then I tought again that it was almost impossible to be the dll... I realy never imagined that the dll would need to change to be able to do this... sry guys, just forget about it xD