Programming
Nova: A JavaScript based template engine for Node.JS
Previously I wrote that I will be releasing a JavaScript based template engine very soon (HOPEFULLY this weekend) for Node.JS, so I wanted to talk a little on that.
One big annoyance with template engines these days for me is the idea that the syntax has to be some special unique format, in which no IDE or editor will ever support out of the box unless it gets extremely huge like Smarty. That to me is very annoying, as you lose all the benefits syntax highlighting and other features your editor can provide.
So I decided to make a template engine where the syntax is simply JavaScript, and the DOM can be controlled fully through JavaScript. Now your site could be fully coded in JavaScript (Client Design, Client Feature, Server, Database (mongo))! JavaScript is on the way to world domination!
The promise is simple. Provide a template system that is 100% JavaScript code (the code is evaluated just like it was a Node.JS module), where your editor can provide out of the box support for syntax highlighting and common errors. Also the ability to lint your template too.
Since the code is fully ran JavaScript, you also have the ability to utilize Node.JS functionality into your template.
Wait you say, “functions and logic inside a template engine?! That goes against the point of a template engine!”
No, It doesn’t. Some random guy on the Internet made a blog post saying HE thinks you shouldn’t have any logic done in a template and it spread, and now everyone thinks that its bad to have logic in a template.
But no, it’s not. You’re end goal is to make use of the tools available to you, and accomplish your goals cleanly, efficiently, and as quickly as possible.
If you only have a flat head screw driver, which fits into the size of a Phillip head screw, would you run to the store to buy a Phillip head screwdriver, or would you use the Flathead and just unscrew the screw?
A tool or library should not necessarily restrict you from doing things. It should be you, the user of said library, who decides the best practices and implementation for your project. If you want to keep all logic out of a template, then do it. If you feel having logic in a template makes your task easier, then do it.
This concept that the tool should dictate what you use it for needs to go.
Nova will be flexible, and not restrictive. So you can build your template however you want. You give it data in its expected fashion, and it will act on that. It’s not going to spank you for using logic in your template. (Although if you’re into that kind of thing I can write a special version for you for a price… :))
Here is an example (very basic) template for Nova:
======
======
as you see, its very simple JavaScript, and since its JavaScript and not JSON, you dont even have to wrap the keys in strings
{h1:{id: ‘header’}}
would work just as fine. Again it’s YOUR choice for standards.
Now. Nova caches templates on load to pure HTML so this syntax isnt parsed EVERY time the template is rendered. But static templates are no fun of course… we need to be able to render dynamic content!
Well, notice that nova variable I didn’t use in the example? That has helper functions! One of those functions is nova.onRender
This function will let you specify a callback function to execute anytime the template is rendered. Note the object notation format is parsed and processed on COMPILE time, ie the very first request, then on render the onRender functions are called to fill in dynamic data.
So how do you use them? take the following template above and lets replace the [‘My content’] with:
=====
=====
So when you call template.render() in your application, you pass it 2 arguments, with the first (optionally) being render variables.
These are variables you can build in your applications logic before rendering to the client, then pass it to template to use just like any other template engine.
The difference here is that since most of the HTML is cached and this is a JavaScript oriented language, the processing and printing of the variables in the template is done through JavaScript functions.
Also, because templates can render Asynchronously.
Notice in the above example, we give a different call to render based on if username was set or not.
Firstly to explain render, anything passed to this function will be the ‘answer’ to what content should fill the onRender spot. So if my username was Aikar, I would see:
<div id="content"> <span>Greetings Aikar</span></div>
Also notice I was able to still use the Nova style syntax of the template, and did not have to type raw HTML!
Nova is extremely flexible, and will give you the tools you need to write fun templates.
Also, again they are Asynchronous. The template runs all onRender async, and will wait for all of them to ‘answer back’
So say you wanted to do some async operation like a database query (again, do what you want with your templates, it’s your choice), you would do something like this:
========
========
Check the Nova docs when its released for more helpers such as partial/include support!
0Giving to the Node.JS community: Nova, Meteor, Wormhole
I’ve done a lot for communities in Final Fantasy XI and other games, and been a good community leader, but now I’m finally ready to release a library to open source! (I’ve worked on projects to release but never finished…)
Over the past week or two I’ve been working on a new Template Engine for Node. Why another you ask? Well cause I felt like it mainly, and wanted to do something interesting.
On our websites we have our client side code in written in JavaScript, and with Node.JS, our server side code is also JavaScript. Then add on the database layer where NoSQL is a more popular choice, and then some of the more popular databases are also founded on… JavaScript! (CouchDB/BigCouch, MongoDB, and maybe sure others?).
So I thought, why not a JavaScript based template engine too!
So now I am working on Nova. As you may know (by either knowing me or reading my blog), I am working on a space game, so I’ve chosen the space theme as a naming scheme for my modules I write. I am wanting to write a lot of the modules Starlis will use, and then release them to the public.
I already have plans to do a MongoDB wrapper similar to Mongoose as my next library, named Meteor. I also have some code I’ve already written a base for message framing and sending over streams, and plan to do an RPC layer on top of, using the MessagePack format for serialization which is a lot better in performance than JSON. It will use pgriess’s node-msgpack library which I’ve forked and added support for Node 0.3.x.
I also plan to add a CSS layer onto Nova shortly after release, providing the same syntax for CSS.
Read my next blog post for details on Nova!
Look forward to my modules soon, First being Nova, then Meteor and Wormhole sometime after.
0Developing MMO Game Servers in PHP and JavaScript
Hello, I’ve just reset my blog and now going to use it for projects going forward, primarily (at this time) Starlis.
During the early months of 2010 I was working on new bleeding edge technologies toying with the idea of a real time browser based game using WebSockets. The idea was fun. Integrate social networking site Twitter into the game and tie in Real World aspects and gameplay elements. I started the game writing the server FULLY in PHP.
I know that makes some people scream.. but I’m rather experienced with PHP and have knowledge in deeper areas of PHP. I’ve worked on an application networking system for a few years that enables multiple PHP processes to communicate to each other, Inter Process Communication.
Why would you do this? Because PHP is single threaded and a synchronous language. DB queries etc block everything. Cant use that TCP connections! So I wrote AppComm based on the idea from a blog post I found on the internet using proc_open() function. It worked great for other projects like IRC bots and RSS feed parsers until I started doing some serious work with it like the idea of a realtime MMO game server with 250 NPC AI clients playing the game at the same time as myself and others…
I started to see massive performance issues, and it was due to the polling nature of the IPC workings of my AppComm system.
So I refactored the AppComm library and all TCP Socket Streams to be stream_socket instead of the socket_ functions, and implemented an event loop around stream_select to monitor all streams and tick functions for timed interval functions. This gives immediate response to data coming into a stream, so the buffers do not get filled up easily (which is what happened on the polling system, tons of data comes in at once and isn’t read until Sleep() is done sleeping.), and enables the process to sleep even longer and not poll at very quick intervals.
This made a huge difference, and the server runs great now. However under heavy load can still get some high CPU and decent lag, but thats due to MySQL queries due to the synchronous nature.
Notice a reoccuring word here thats a problem? Synchronous…
So I then found Node.JS… Boy does it solve that problem. I toyed with the idea of a rewrite of the game I was working on, but was hesitant about time since already got pretty far into the game and was now at the point of implementing game play elements.
Interestingly, I believe PHP would be decent under my AppComm system without the slowness of MySQL, as that was really the main place CPU time was spent, but I know it’ll never be as efficient as an async system like Node.JS.
But things got busy and dev stopped for a while on the PHP server. Then one day I came up with a great idea for a different game, a space game where I wouldn’t be bound to the horrors of 3d mathematics for handling the shape of the earth (original game was built on top of Google Maps) in all the math calculations.
Lets cut to the chase, I’m working on projects in my free time cause I want to make money off them. With this Twitter interaction game I really could not see many ways to monetize it. I thought of some, but it relied on partnership with advertising companies and other companies etc. I couldn’t see many ways to directly take money from the players.
That’s why my new idea, which I named Starlis, got me so excited. I was out of the grasps of real world ties, and now in a fantasy world freely open to my own imagination. Lets just say I have a very creative imagination.
I immediately started seeing ways to monetize the game, which gives huge motivation to work on the game and make the game good… Because you know the better the game is, the more people play it, the more people will pay me.
Now I can start over in Node.JS! I learned a good bit from the first initial game idea in PHP, and from my own personal experience in gaming: Today’s programmers are idiots. Performance is important.
Sadly I hear so too often to ignore performance from the start and come back and improve it only if its a problem. I strongly disagree with that ideology and refuse to accept it. I have C++ experience with game hacking, which involved ASM and deep memory work, function hooking etc, so I have a functional understanding on the computational flow path of code, regardless of language.
It’s not hard to see where bottlenecks can occur on your program even before you write it.. I firmly believe taking time to design and plan out an applications architecture will overall save you time.
So that’s exactly what I’m doing. Thinking of the areas of code that can be bottlenecks, and writing it efficiently from the start. areas like a network protocol parser will be called in almost every step of the app, so that need to be written efficiently.
I’m currently designing and building the Starlis Server Architecture. It’s designed around full Asynchronous IO, and designing a Complete Graph server architecture with each node running copies of the same code on different physical servers, and each includes a Message Routing system that will keep track of every server in the networks load levels, and automatically send the message to whichever server has the lowest load.
Designing this simple idea from the start and laying it the foundation will be adding horizontal scaling to the application before the application is even written. no hacking or modifying code after the games launched and having down time due to crashes and maintenance due to the re-engineering.
It’s harder to implement a better foundation for a house when a house is on the lot, than it is when there is no house…
You never know if the changes you have to make to achieve scalability requires massive fundamental changes to your applications logic, which if it does will likely cause you to do the least work possible. Around here we call that half-assing. I’m not fond of half-assing my work.
So with Node.JS, I’m developing a fully scalable application architecture from the get go to handle the games server, and top that with MongoDB for its extremely fast performance and built in amazing sharding support, we have a solid foundation for Starlis. A foundation that says “I’m ready for anything.” support for vertical AND horizontal scaling, with code that could be written in 1 week (I’m developing this on my free time, so other RL commitments and personal entertainment makes it impossible to accurately judge how long something is taking me).
I personally think that’s awesome.
2About Aikar
I am Senior Software Engineer and Entrepeneur. I am an enthusiast and love creating things. I operate my own side company in my free time called Starlis LLC, working in Minecraft.
I enjoy doing things right and learning modern technologies.