“Nine people can’t make a baby in a month.” Fred Brooks

“Weeks of coding can save you hours of planning.” unknown

You can find a lot more here

 Share now!

In order to understand what changed, i recommend you to read the great resume done in Axod’s Hack .

Mainly you need to get 3 keys, 2 of them are send in the headers and the 3rd is sent after the headers. Then you need to do some “weird” transformation with the 3 and send them back after the header.

After having the 3 keys, here is the code in erlang to build the challenge to send back to client:

Now that you have the challenge just build the response like this :

Challenge = build_challenge(Key1,Key2,Key3),

HandshakeServer = [
        "HTTP/1.1 101 Web Socket Protocol Handshake\r\n",
	"Upgrade: WebSocket\r\n",
	"Connection: Upgrade\r\n",
	"Sec-WebSocket-Origin: ", Origin , "\r\n",
	"Sec-WebSocket-Location: ws://", Location , "\r\n\r\n",
	Challenge
],

As soon i can i will create a new branch on misultin web server so you can see the code working.

 Share now!

(af83@dakua.local)34> timer:tc(proplists,get_value,[99999,List]).
{8685,99999}
(af83@dakua.local)35> timer:tc(lists,keyfind,[99999,1,List]).
{829,{99999,99999}}

WOW! for a list of 100.000 elements proplists:get_value can be 10x slower then lists:key_find!

So now i’m using this encapsulation in order to maintain proplists syntax and avoid extra typing

proplist_get_value(Key,List)->
  case lists:keyfind(Key,1,List) of
    {_K,V}->
      V;
    false->
      undefined
  end.

Now the question is Why? why lists is much more faste then proplists? Our guess is that since lists functions are BIF’s, they must be build in C and proplists no, but this is just a guess, if someone has a better explanation please share.

 Share now!

 Share now!

It accepts any kind of of url, retrieving zero or one redirect location.

Just copy the code and give it a test :

darkua:~ sergioveiga$ erl
Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false]

Eshell V5.7.4 (abort with ^G)
1> c(unshort_srv).
{ok,unshort_srv}
2> unshort_srv:start_link().
{ok,<0.39.0>}
3> unshort_srv:unshort(”http://tarpipe.com/kE”).
{ok,”http://sergioveiga.com/index.php/2010/02/21/erlang-code-to-unshort-urls/”}
4> unshort_srv:unshort(”http://google.com”).
{ok,”http://www.google.com/”}
5>

 Share now!

“Comparisons between Ericsson-internal development projects indicate similar line/hour
productivity, including all phases of software development, rather independently of which
language (Erlang, PLEX, C, C++ or Java) was used. What differentiates the different
languages then becomes source code volume.

Some concrete comparisons of source code volume have been made, as applications
written in C++ have been rewritten in Erlang, resulting in a ten-fold reduction in the
number of lines of uncommented source code. Other comparisons have indicated a four-
fold reduction. A reasonable conclusion is that productivity increases by the same factor,
given the same line/hour programmer productivity.

When measuring product quality in number of reported errors per 1000 lines of source
code, the same relationship seems to exist: similar error density, but given the difference
in code volume, roughly 4-10 times fewer errors in Erlang-based products.
Furthermore, experiences from maintaining some Erlang-based products in the field
indicates that error corrections usually do not give rise to unwanted side-effects. This is in
line with what one would expect from using a mostly side-effect free declarative
language.

These comparisons do not pretend to hold up to scientific scrutiny, but they are collected
from indicators which show a rather consistent pattern. They also seem consistent with
opinions offered by non-Ericsson developers [one2one],[sendmail].
While AXD 301 consists of programs written in C, C++ and Java as well as Erlang, and
each language is used where it fits best, practically all the complex work is done in
Erlang. This is usually decided by the designers themselves – many of our most skilled C
programmers prefer Erlang for complex tasks, and consider it the obvious choice for
anything that involves concurrency or distribution.

Finally, we have seen that programmers – even those with no previous exposure to
functional languages – learn Erlang quickly, and become productive within a matter of
weeks.”

I had exactly the same feeling when started to code erlang comparing to my previous most used languages, Java and  PHP.

 Share now!

So mainly its similar to meebo/mashable integration, but without needing to register in any place, and much more simpler.

The best part of this idea, to me, is that would be a great project to push erlang to the max

What you think?

 Share now!

It will be a simple introduction to erlang, and how you can use it for building web applications. With erlang and nitrogen you can very easily start building real-time web applications, using comet, that are fault-tolerant. I will assume that you have never heard of erlang before, so don’t be afraid to join us.

The talk will follow the tutorial i published some hours ago , so i you are attending i suggest a quick read, in order to make the talk more productive!

 Share now!

Tools & Tips

Toolbar : a very simple visual toolbar for the most used tools in erlang. Type on your erlang shell toolbar:start(). You should see 4 icons representing this tools:

• Table Viewer: An ETS and MNESIA graphical table visualizer. With this you can view and change data inside a ets or mnesia table. On shell you can call it using tv:start().

• Process Manager : A process manager used to inspect the state of an Erlang system. Call on erlang shell : pman:start(). X11 window will open showing the running processes for this node.

• Debugger : A debugger for debugging and testing of Erlang programs. Call on erlang shell : debugger:start(). A window will open like this one

• Application Monitor : A utility used to supervise Applications executing on several Erlang nodes. Very useful when you start deploying cluster of application in different nodes On shell use appmon:start().

Some other useful tips:

Tip1 : if you have no internet to search, you can use erlang manual. Just type this in your terminal

erl -man lists

Tip2 : in order to let an application running in background, all you need is to add this to your start script

-detached \

Tip3 : you probably have notice that we are using error_logger module. This is the best way to deal with errors, and for example get warned by email each time your application is in trouble. Please read more here

Applications

As you probably noticed i did not talk about any data-store mechanisms, in order to avoid to much detail. In erlang there are many options to store data. Erlang has two system modules ets and dets that are used for efficient storage of large numbers of Erlang terms. ETS is short for Erlang term storage,
and DETS is short for disk ets. ETS and DETS perform basically the same task: they provide large key-value lookup tables. ETS is memory resident, while DETS is disk resident. You can use ETS or DETS directly or you can use erlang database : MNESIA, a distributed DataBase Management System, with a relational/object hybrid data model. Mnesia is really cool because it maps directly to records, and you have a nice query language to it.

Another great way to store info in erlang is using one of the coolest erlang projects: couchdb is a document-oriented database that can be queried and indexed in a MapReduce fashion using JavaScript. In other words you don’t need a schema to your data, and since its json its very easy to write/read. This is very useful for example when you are saving 3rd party data that you don’t control, like tweets. In order to make the interaction between erlang and couchdb or any other service, before you start coding, look first at github (another great project using erlang), where you can find a lot erlang code, that will make you life even easier.

Some useful Erlang Resources

You can find a lot of stuff on the web about erlang, but here are some of our favorite bookmarks about erlang :

• http://erlang.org/doc/man/
• http://nitrogenproject.com/
• http://trapexit.org/
• http://www.planeterlang.org/
• http://blog.socklabs.com/
• http://yarivsblog.com/
• http://spawnlink.com/
• http://medevyoujane.com/
• http://metajack.im/
• http://anders.conbere.org/blog/
• http://kungfooguru.wordpress.com/
• http://beebole.com/erlang/en/
• http://erlanganswers.com/web/mcedemo
• http://schemecookbook.org/Erlang/AllRecipes
• http://www.undefined.org/c4-1/slides/slides.html
• http://jeremy.marzhillstudios.com/
• http://www.infoq.com/presentations/Systems-that-Never-Stop-Joe-Armstrong

Resume

Erlang it has been around for a long time now, but the buzz around erlang started around 2007 with the publication of Programming Erlang: Software for a Concurrent World by Joe Armstrong, erlang creator and the use of erlang in really projects, like ejabberd, couchdb or facebook chat system. With this buzz erlang community start to grow outside the Telecom world specially in the web world.

Erlang is not just another programming language,  for me that have been programming more then 5 years with OO, its new paradigm, a new way to think, a new form of coding. And we are deeply in love with it… we hope you come to.

If you are not yet sold, just watch erlang demo video from the 80’s and you will for sure

 Share now!

Clone this git repositorium :

darkua:~ sergioveiga$ git clone git://github.com/darkua/ErlanOnNitro_TwitterStream.git
Initialized empty Git repository in /Users/sergioveiga/ErlanOnNitro_TwitterStream/.git/
remote: Counting objects: 72, done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 72 (delta 4), reused 0 (delta 0)
Receiving objects: 100% (72/72), 144.83 KiB | 182 KiB/s, done.
Resolving deltas: 100% (4/4), done.
darkua:~ sergioveiga$ cd ErlanOnNitro_TwitterStream
darkua:ErlanOnNitro_TwitterStream sergioveiga$ ./start.sh

NOTE : in this project we will be use a lib packed with mochiweb project, so please check if already you have mochiweb intalled. (say thanks to @bpedro, for pointing this out )

twitter_stream@localhost)4> mochijson2:encode(<<"hello">>).
[34,<<"hello">>,34]

if you don’t see this, you need to install mochiweb first. Its very easy:

darkua:twitterstream sergioveiga$ echo $ERL_LIBS
/opt/local/lib/erlang/lib/:/usr/lib/erlang/lib/:/Users/sergioveiga/erlangProjects/tedi
darkua:twitterstream sergioveiga$ cd /opt/local/lib/erlang/lib/
darkua:lib sergioveiga$ sudo svn checkout http://mochiweb.googlecode.com/svn/trunk/ mochiweb-read-only
darkua:lib sergioveiga$ cd mochiweb-read-only/
darkua:mochiweb-read-only sergioveiga$ make

Now you probably need to kill erlang shell, just hit ctr-c twice. Start it again and now mochijson should already be available.

Before you go and check your browser you need to enter your twitter credentials, needed for twitter stream. So please go to include/include.hrl and add your own credentials, and compile the project.

Make sure you dont any other server running on port 8000, and go to your browser http://localhost:8000/. Here you will find some descriptions and links to both examples we will follow in this part.

Now choose you favorite IDE, we are using textmate, and open the project dir.

So this is ver similar with your previous nitrogen project, with some little changes. Open your project app file inside ebin dir. Here we have added templateroot so we can put all templates inside a different directory.

We have created an include dir, where we will put all our include files, and on src, we divide the pages, from the libs, that will hold “hard-core” code.

Example 1 – Twitter Stream

A screenshot of how twitter stream looks like :

Now open /src/pages/web_index.erl. You can see that we almost don’t do nothing here, because this is a static file, so we are only are adding the page title, and saying witch template will be used to render the page. All the code you can see its on index.html template file.

Now open web_stream page.

Lets check body function

body() ->

 %%builds the stream
 Stream = start_stream(),

 %% the html to render
 Body= [
 #panel{class="header", body=[
 #h1{text="Erlang on Nitro - Twitter Stream"},
 #panel{id=feedback,class="feedback", body=[
 "Calling Twitter...pls wait..."
 ]},
 #panel{id=control,body=[#button{text="Stop", postback={stop,Stream}}]}
 ]},
 #panel{id=stream,body=[]}
 ],

 wf:render(Body).

Here we are calling stream_start() to open stream connections and building the page html structure. Now check the start_stream() to see where is the magic.

start_stream()->
 Pid = wf:comet(fun()-> ?MODULE:loop() end),

%%check for track
 case wf:get_path_info() of
 []->
 twitter_stream:sample(Pid);
 Track->
 twitter_stream:track(Track,Pid)
 end.

When calling wf:comet, we are building a comet connection to this page, and passing what function will be used to “comunicate” with the browser. Then we use wf:get_path_info() to get extra info on path, that we will use to call twitter, for example the twitter or hello keyword from the examples. All the connection to twitter stream api happens inside twitter_stream, and for now you can look at it as a black box that streams tweets for you, later you can check it out and try to understand what is happening.

Now before checking loop function, its time to better undersand one of erlang top features, creating and communicating with processes. When we did wf:comet, what is happening in resume, is that we have spawned loop function into a new process, and now we can comunicate with it by sending to the Process Id messages. And to do that we just need to do this :

Pid ! message

In order to put a process waiting for messages is also very simple, just do this :

receive
 Message->
 ...
 end.

When creating a receive block, you are telling erlang to wait there until he receives a new message, that he will try to match to execute the respective code.

Now lets check a very simple example in order to better understand this. Open file spawn_example inside libs. You can see that we just have 2 functions, start and loop. On start we are spawning function loop, and registering the process with a name to be easier to call them after, like your fathers did with you

Now go to erlang shell and call this:

(twitter_stream@localhost)2> spawn_example:start().
true
(twitter_stream@localhost)3> bot ! hello.
Hello. How are you?
hello
(twitter_stream@localhost)4>

Very simple and cool, but now try to call it again

(twitter_stream@localhost)15> bot ! hello.
** exception error: bad argument
 in operator  !/2
 called as bot ! hello
(twitter_stream@localhost)16>

Humm… the reason it failed its very simple. After the process received the first message he executed the code matched to hello and it ended. So when you try to send a new message to it, it no longer exists, and it fails. So if you want to create process that after processing a message continues waiting for other you need to recall the function. So in the end of the loop add a call to loop.

loop()->
 receive
 hello ->
 io:format("Hello. How are you?~n",[]);
 _->
 void
 end,
 loop().

Now lets try again

(twitter_stream@localhost)16> sync:go().
Recompile: ./src/libs/spawn_example
ok
(twitter_stream@localhost)17> spawn_example:start().
true
(twitter_stream@localhost)18> bot ! hello.
Hello. How are you?
hello
(twitter_stream@localhost)19> bot ! hello.
Hello. How are you?
hello

As you can see now the process no longer dies. Now lets see another great feature of erlang: hot code swapping, the ability to change the code without stopping the system. Try to change the text on the message, for example to:

io:format("Hot swapping rocks!~n",[])

sync:go and lets see what happens:

(twitter_stream@localhost)24> sync:go().
Recompile: ./src/libs/spawn_example
ok
(twitter_stream@localhost)25> bot ! hello.
Hello. How are you?
hello

Humm… the old message is still there. The reason this happens is very simple. If we dont call “externally” by  prefixing the module name we are saying to erlang to “use this version of this method” and not the current version. So all we need to do is add the ?MODULE to the loop call, in the start function and also in the end of the loop. Now sync your code, and start your bot again.

(twitter_stream@localhost)38> sync:go().
Recompile: ./src/libs/spawn_example
ok
(twitter_stream@localhost)39> spawn_example:start().
true
(twitter_stream@localhost)40> bot ! hello.
Hot swapping rocks!
hello

Now lets change the message again, to “Hot swapping really rocks”. Sync your code again and send him a message.

(twitter_stream@localhost)41> sync:go().
Recompile: ./src/libs/spawn_example
ok
(twitter_stream@localhost)42> bot ! hello.
Hot swapping rocks!
hello
(twitter_stream@localhost)43> bot ! hello.
Hot swapping really rocks!
hello

As you can see the first message the bot receives the old code will be executed, because the process was already blocked waiting for a message, but when it run again it will already be using the current version of the code. You can see final version of spawn_example here.

Now back to our example, you can see exactly this happening with the loop function inside web_stream module. As you can see the loop function will be waiting for a subset of messages that twitter stream will send to him, and according to each we will be doing something different. Now what is important to understand is how nitrogen communicates with the client browser.

We are using 2 nitrogen functions  wf:update(id, content) and wf:insert_top(id, content) that will update and insert on the top of a DOM element. Nitrogen will translate this to the corresponding javascript, using jquery lib, and send it to the browser when we call wf:comet_flush(). So only when call this function inside a comet loop, we will be sending all previous javascript code built with wf:update/wf:insert_top” as a response to the comet connection.

loop()->
 receive
 {unauthorized,Message}->
 io:format("~p~n",[Message]),
 wf:update(feedback,"Your twitter credentials are wrong, check include.htr file!");
 {start,_Message}->
 wf:update(feedback,"Stream Running...");
 {stream,Message}->
 get_tweet_info(Message);
 {stream_end,_Message}->
 restart_button(),
 wf:update(feedback,"Stream Ended...");
 {stop,_Reason}->
 restart_button(),
 wf:update(feedback,"Stream Stoped...");
 {error,Reason}->
 error_logger:error_msg("Error : ~p~n",[Reason]),
 restart_button(),
 wf:update(feedback,#panel{class="error",body=wf:f("Error : ~s !",[Reason])});
 Any->
 error_logger:info_msg("ANY : ~p~n",[Any])
 after 55000 ->
 wf:update(feedback,"to quiet for me...")
 end,
 wf:comet_flush(),
 ?MODULE:loop().

With this simple code you have learn the basic blocks to create parallel computing, and the possibility to push content to users without the need to be constantly polling a service. And this is really cool because is all you need to create cool real-time / multi-user applications.

Example 2 – Twitter Cloud

Twitter Cloud Screenshot :

Nitrogen does already lot of work for us on erlang/javascript interaction, making easier  the creation of appealing  and rich web applications. Check web_cloud example where we have created another visualization for twitter stream, but now its a cloud of twitter users, where the size of the pictures is related to the ratio following/followers, where you can click to see the status update.

if you open web_cloud.erl, you can see that its very closed to web_stream, but now instead of building the html on the server we are passing the info to the the browser and will will build the dom with javascript. This can be achieved very easily with nitrogen using wf:wire constructor. As you can check on line 126 of web_cloud

wf:wire(wf:f("buildTweet({id:'~s',screen_name:'~s',picture:'~s',text:\"~s\",size:'~p'})",
[Tweet#tweet.id,
Tweet#tweet.screen_name,
Tweet#tweet.picture,
wf_utils:js_escape(Tweet#tweet.text),
Tweet#tweet.size])
).

Mainly we are calling the js function buildTweet that you can find on /js/main.js a js object with all the tweet function. We also use another great nitrogen function wf:f, that helps you build complex strings.

Another thing really useful is giving to an event different actions, like you can see in line 27.

#panel{id=control,body=[#button{text="Stop", actions=#event{actions="$.growl('','Stream is Stoping...');"}, postback={stop,Stream}}]}

Inside each event you can define your postback, and you can also use actions parameters to execute javascript. So if you click on the stop button, we will be calling event({stop,Stream}) and also calling jquery-growl plugin that will show a message to the user.  We can even define more than one event type to the same element. For example change control panel this on body function

#panel{id=control,body=[#button{text="Stop", actions=[

#event{actions="$.growl('','Stream is Stoping...');", postback={stop,Stream}},

#event{type=mouseover,actions="$.growl('','Click if you are a mouse!...');"}]
}]}

sync:go, and check your example. You should be seeing a message appear when you put your mouse over stop button and other when you click it.
As you can imagine this is a very simple example of all you can do with nitrogen . So please check nitrogen project where will have a lot of cool examples ,  nitrogen api with all functions available, and you can check some more documentation on on github wiki.

So now you know already enough to start coding your own ideas and projects, but before read last part of this tutorial, where we will resume some of the tools erlang has to help on your adventure.

 Share now!