Starlis

Get sucked in… Wormhole!

April 17, 2011 in Node.JS, Programming, Projects, Starlis, Wormhole

I have finally released my 2nd module to the Node.JS community, Wormhole!

Wormhole provides a message queue system over network streams using the MessagePack library for serialization. The streaming deserializer is great for performance and memory usage, and allows you to have very high performance compared to JSON or traditional message queues.

Check out the Wormhole library on github

0

Nova: A JavaScript based template engine for Node.JS

January 21, 2011 in Node.JS, Nova, Programming, Projects, Starlis

Previously I wrote that I will be releasing a JavaScript based template engine very soon (HOPEFULLY this weekend) for Node.JS, so I wanted to talk a little on that.

One big annoyance with template engines these days for me is the idea that the syntax has to be some special unique format, in which no IDE or editor will ever support out of the box unless it gets extremely huge like Smarty. That to me is very annoying, as you lose all the benefits syntax highlighting and other features your editor can provide.

So I decided to make a template engine where the syntax is simply JavaScript, and the DOM can be controlled fully through JavaScript.  Now your site could be fully coded in JavaScript (Client Design, Client Feature, Server, Database (mongo))! JavaScript is on the way to world domination!

The promise is simple. Provide a template system that is 100% JavaScript code (the code is evaluated just like it was a Node.JS module), where your editor can provide out of the box support for syntax highlighting and common errors. Also the ability to lint your template too.

Since the code is fully ran JavaScript, you also have the ability to utilize Node.JS functionality into your template.

Wait you say, “functions and logic inside a template engine?! That goes against the point of a template engine!”

No, It doesn’t. Some random guy on the Internet made a blog post saying HE thinks you shouldn’t have any logic done in a template and it spread, and now everyone thinks that its bad to have logic in a template.
But no, it’s not. You’re end goal is to make use of the tools available to you, and accomplish your goals cleanly, efficiently, and as quickly as possible.

If you only have a flat head screw driver, which fits into the size of a Phillip head screw, would you run to the store to buy a Phillip head screwdriver, or would you use the Flathead and just unscrew the screw?

A tool or library should not necessarily restrict you from doing things. It should be you, the user of said library, who decides the best practices and implementation for your project. If you want to keep all logic out of a template, then do it. If you feel having logic in a template makes your task easier, then do it.

This concept that the tool should dictate what you use it for needs to go.

Nova will be flexible, and not restrictive. So you can build your template however you want. You give it data in its expected fashion, and it will act on that. It’s not going to spank you for using logic in your template. (Although if you’re into that kind of thing I can write a special version for you for a price… :))

Here is an example (very basic) template for Nova:

======

======

as you see, its very simple JavaScript, and since its JavaScript and not JSON, you dont even have to wrap the keys in strings

{h1:{id: ‘header’}}

would work just as fine. Again it’s YOUR choice for standards.

Now. Nova caches templates on load to pure HTML so this syntax isnt parsed EVERY time the template is rendered. But static templates are no fun of course… we need to be able to render dynamic content!

Well, notice that nova variable I didn’t use in the example? That has helper functions! One of those functions is nova.onRender
This function will let you specify a callback function to execute anytime the template is rendered. Note the object notation format is parsed and processed on COMPILE time, ie the very first request, then on render the onRender functions are called to fill in dynamic data.

So how do you use them? take the following template above and lets replace the [‘My content’] with:

=====

=====

So when you call template.render() in your application, you pass it 2 arguments, with the first (optionally) being render variables.
These are variables you can build in your applications logic before rendering to the client, then pass it to template to use just like any other template engine.

The difference here is that since most of the HTML is cached and this is a JavaScript oriented language, the processing and printing of the variables in the template is done through JavaScript functions.
Also, because templates can render Asynchronously.

Notice in the above example, we give a different call to render based on if username was set or not.
Firstly to explain render, anything passed to this function will be the ‘answer’ to what content should fill the onRender spot. So if my username was Aikar, I would see:

<div id="content">  <span>Greetings Aikar</span></div>

Also notice I was able to still use the Nova style syntax of the template, and did not have to type raw HTML!
Nova is extremely flexible, and will give you the tools you need to write fun templates.

Also, again they are Asynchronous. The template runs all onRender async, and will wait for all of them to ‘answer back’

So say you wanted to do some async operation like a database query (again, do what you want with your templates, it’s your choice), you would do something like this:

========

========

Check the Nova docs when its released for more helpers such as partial/include support!

0

Giving to the Node.JS community: Nova, Meteor, Wormhole

January 21, 2011 in Meteor, Node.JS, Nova, Programming, Projects, Starlis, Wormhole

I’ve done a lot for communities in Final Fantasy XI and other games, and been a good community leader, but now I’m finally ready to release a library to open source! (I’ve worked on projects to release but never finished…)

Over the past week or two I’ve been working on a new Template Engine for Node. Why another you ask? Well cause I felt like it mainly, and wanted to do something interesting.

On our websites we have our client side code in written in JavaScript, and with Node.JS, our server side code is also JavaScript. Then add on the database layer where NoSQL is a more popular choice, and then some of the more popular databases are also founded on… JavaScript! (CouchDB/BigCouch, MongoDB, and maybe sure others?).

So I thought, why not a JavaScript based template engine too!

So now I am working on Nova. As you may know (by either knowing me or reading my blog), I am working on a space game, so I’ve chosen the space theme as a naming scheme for my modules I write. I am wanting to write a lot of the modules Starlis will use, and then release them to the public.

I already have plans to do a MongoDB wrapper similar to Mongoose as my next library, named Meteor. I also have some code I’ve already written a base for message framing and sending over streams, and plan to do an RPC layer on top of, using the MessagePack format for serialization which is a lot better in performance than JSON. It will use pgriess’s node-msgpack library which I’ve forked and added support for Node 0.3.x.

I also plan to add a CSS layer onto Nova shortly after release, providing the same syntax for CSS.

Read my next blog post for details on Nova!

Look forward to my modules soon, First being Nova, then Meteor and Wormhole sometime after.

0

Starlis: Bridging the gap between Social Casual, Casual, and Hardcore

December 29, 2010 in Starlis

In the early days of gaming, there was just 1 category: Gamers.
But then World of Warcraft came… Everything had changed. WoW introduced a game that was fun and playable even for people that have very little time.

This introduced a new category of gamers: Casuals, relabeling the existing group to ‘Hardcore’.
WoW bridged the gap between Casual and Hardcore and did it greatly, although some of the extreme Hardcore players may disagree.

Then comes Facebook with its platform, and the company Zynga. Games like FarmVille now have brought another form of people into the gaming world. People that enjoy the idea of logging in once a day for only 10 minutes at a time to do some repetitive task and help their friends, then log off.

I would refer to them as “Social Casual” players.

So now we have 3 groups of players, with all varying levels of depth into video games.

I myself would be in between Hardcore and Casual. I love games, and I’m fully Hardcore at heart, but just like any other successful adult in the world, I have a job which limits my time, so that I can not play games at a Hardcore level anymore. I enjoy WoW but I miss the challenge and long term goals that come in other non casual friendly games.

People like myself, and above me in the hardcore category find it hard to get into the Zynga type games such as FarmVille and FrontierVille. It’s just too corny, there’s not enough depth.

However, I believe it is possible to bridge the gap between all 3. I have a lot of experience with various forms of gaming. I use to be full on hardcore, unemployed playing Final Fantasy XI for 18 hours a day. So I know what I loved about those times. I also have experience in the casual world, knowing what aspects of WoW is appealing to casuals with limited game time since I now myself am in that category. And then there is FarmVille. I’m a programmer so I can analyze various aspects of a game and understand how it clicks with people. I played FarmVille for a little bit to see how the games UI worked, what the daily overall gameplay experience was, etc.

So now, to take those concepts, and bring them into a universe that Casual and Hardcore players also play in.

It’s easy to make a Zynga knock off, but as I said, people like myself find those games corny and without depth. So the goal is to keep the same concepts of a Zynga game, but keep it mature enough so that a hardcore player will enjoy it too.

Casual and Social Casual really aren’t that far off from each other, just that the casual would play longer and play more in depth games, but they have 1 main concept in common: Play simple, rewarding games for fun to waste time.

So developing a casual friendly UI can be tricky to pull off and not make your hardcore users run away too.
So the UI needs to be simple, self explanatory but still provide help menus. Why self explanatory and help menus? Well Hardcore players tend to jump in and want to understand it just by looking at it without looking at help. Some casuals may agree with it too. But not everyone’s brain works the same way and may still need detailed help.

WoW was friendly to players in that it ensured you were taught how to play the game, even when the game was pretty easy to figure out without help menus.

Also, the UI and the game in general can not be as ‘cheerful’ as Zynga games are. Players like myself see it as corny and childish, and it really distracts you from the seriousness of the game.
Sadly, that artistic style is an attraction point for some of Zynga’s users, but that is a sacrifice I will have to make in order to keep the hardcore players interested.

The way I intend to bridge the game is to have 2 major UI portions of the game.

  1. Inside the space station:
    • This is going to be the primary “Social Casual” and Casual UI.
    • This UI will manage the Economic aspect of the game, including resource gathering, processing, production, research and market based activities.
    • A Social Casual player would likely never leave the space station. They will log in daily to order their workers to work, research new technologies, and execute build orders to produce ships and items.
    • Players will then be able to sell said items on the market to make money, and buy other items
    • More ideas on inside space station I do not want to make public at this time… but lets say they are major.
  2. Outside the space station:
    • This is the more serious part of the game, not designed for Social Casual players.
    • This is going to be a full on RTS style gameplay, where players start in a protected galaxy. However to bridge the casuals into the universe, players will be able to fly their ships inside this galaxy but will not be able to attack players. They will be limited to the 1 space station they start with in the protected galaxy, so they can play around with the idea of flying in space with what they create.
    • Once the player exits the protected galaxy, they enter the real game where players can fight each other and brings in the full RTS gameplay style.
    • This is where the main gameplay for hardcore players will be, and even casuals. The gameplay will not really require long login times, but it will not be limited to the aspects of a reward without punishment environment the space stations in the protected galaxy provide. So a casual player can still enjoy the experience, or stay in the protected areas if they so desire and participate in special PVP events only or PVE. NPC’s in PVE events in the protected systems will not be aggressive, so players are opt-in to fighting PVE in protected areas, but in other galaxies all PVE NPC’s are aggressive and may attack.

So with this, we have a safe and rewarding experience for the social casuals who do not care for anything over simplicity, we have an experience for casuals who either want the simple gameplay or the freedom to opt in the more aggressive gameplay ,and we have the aggressive gameplay for the hardcore.

(Almost) everyone can be happy. I will likely be excluding the people who want the childish UI elements, and don’t want to play anything that has a ‘serious’ feel to it, but that is a needed sacrifice in order to make an overall good game.

2

Developing MMO Game Servers in PHP and JavaScript

December 29, 2010 in Programming, Starlis

Hello, I’ve just reset my blog and now going to use it for projects going forward, primarily (at this time) Starlis.

During the early months of 2010 I was working on new bleeding edge technologies toying with the idea of a real time browser based game using WebSockets. The idea was fun. Integrate social networking site Twitter into the game and tie in Real World aspects and gameplay elements. I started the game writing the server FULLY in PHP.

I know that makes some people scream.. but I’m rather experienced with PHP and have knowledge in deeper areas of PHP. I’ve worked on an application networking system for a few years that enables multiple PHP processes to communicate to each other, Inter Process Communication.

Why would you do this? Because PHP is single threaded and a synchronous language. DB queries etc block everything. Cant use that TCP connections! So I wrote AppComm based on the idea from a blog post I found on the internet using proc_open() function. It worked great for other projects like IRC bots and RSS feed parsers until I started doing some serious work with it like the idea of a realtime MMO game server with 250 NPC AI clients playing the game at the same time as myself and others…

I started to see massive performance issues, and it was due to the polling nature of the IPC workings of my AppComm system.

So I refactored the AppComm library and all TCP Socket Streams to be stream_socket instead of the socket_ functions, and implemented an event loop around stream_select to monitor all streams and tick functions for timed interval functions. This gives immediate response to data coming into a stream, so the buffers do not get filled up easily (which is what happened on the polling system, tons of data comes in at once and isn’t read until Sleep() is done sleeping.), and enables the process to sleep even longer and not poll at very quick intervals.

This made a huge difference, and the server runs great now. However under heavy load can still get some high CPU and decent lag, but thats due to MySQL queries due to the synchronous nature.

Notice a reoccuring word here thats a problem? Synchronous…

So I then found Node.JS… Boy does it solve that problem. I toyed with the idea of a rewrite of the game I was working on, but was hesitant about time since already got pretty far into the game and was now at the point of implementing game play elements.

Interestingly, I believe PHP would be decent under my AppComm system without the slowness of MySQL, as that was really the main place CPU time was spent, but I know it’ll never be as efficient as an async system like Node.JS.

But things got busy and dev stopped for a while on the PHP server. Then one day I came up with a great idea for a different game, a space game where I wouldn’t be bound to the horrors of 3d mathematics for handling the shape of the earth (original game was built on top of Google Maps) in all the math calculations.

Lets cut to the chase, I’m working on projects in my free time cause I want to make money off them. With this Twitter interaction game I really could not see many ways to monetize it. I thought of some, but it relied on partnership with advertising companies and other companies etc. I couldn’t see many ways to directly take money from the players.

That’s why my new idea, which I named Starlis, got me so excited. I was out of the grasps of real world ties, and now in a fantasy world freely open to my own imagination. Lets just say I have a very creative imagination.
I immediately started seeing ways to monetize the game, which gives huge motivation to work on the game and make the game good… Because you know the better the game is, the more people play it, the more people will pay me.

Now I can start over in Node.JS! I learned a good bit from the first initial game idea in PHP, and from my own personal experience in gaming: Today’s programmers are idiots. Performance is important.

Sadly I hear so too often to ignore performance from the start and come back and improve it only if its a problem. I strongly disagree with that ideology and refuse to accept it. I have C++ experience with game hacking, which involved ASM and deep memory work, function hooking etc, so I have a functional understanding on the computational flow path of code, regardless of language.

It’s not hard to see where bottlenecks can occur on your program even before you write it.. I firmly believe taking time to design and plan out an applications architecture will overall save you time.

So that’s exactly what I’m doing. Thinking of the areas of code that can be bottlenecks, and writing it efficiently from the start. areas like a network protocol parser will be called in almost every step of the app, so that need to be written efficiently.

I’m currently designing and building the Starlis Server Architecture. It’s designed around full Asynchronous IO, and designing a Complete Graph server architecture with each node running copies of the same code on different physical servers, and each includes a Message Routing system that will keep track of every server in the networks load levels, and automatically send the message to whichever server has the lowest load.

Designing this simple idea from the start and laying it the foundation will be adding horizontal scaling to the application before the application is even written. no hacking or modifying code after the games launched and having down time due to crashes and maintenance due to the re-engineering.

It’s harder to implement a better foundation for a house when a house is on the lot, than it is when there is no house…

You never know if the changes you have to make to achieve scalability requires massive fundamental changes to your applications logic, which if it does will likely cause you to do the least work possible. Around here we call that half-assing. I’m not fond of half-assing my work.

So with Node.JS, I’m developing a fully scalable application architecture from the get go to handle the games server, and top that with MongoDB for its extremely fast performance and built in amazing sharding support, we have a solid foundation for Starlis. A foundation that says “I’m ready for anything.” support for vertical AND horizontal scaling, with code that could be written in 1 week (I’m developing this on my free time, so other RL commitments and personal entertainment makes it impossible to accurately judge how long something is taking me).

I personally think that’s awesome.

2

Recent Comments

About Aikar

I am Senior Software Engineer and Entrepeneur. I am an enthusiast and love creating things. I operate my own side company in my free time called Starlis LLC, working in Minecraft.

I enjoy doing things right and learning modern technologies.