helion-prime
home about us blogs contacts

Archive for the ‘web-development’ Category

Fast scalability of Ruby on Rails with mongrel under OpenBSD

Wednesday, August 12th, 2009

preamble

Everybody who works with ruby on rails more then month knows at least 2 things:
it’s great framework, but can’t handle multiple requests simultaneously due to it still doesn’t use threads.

fast start

As scalability is common issue, rubyonrails site wiki provides full set of solutions:
[http://wiki.rubyonrails.org/#deployment_stacks]

But we as fast solution will use great OpenBSD pf (Packet Filter) that capable to do many cool things.
First issue is standard mongrail cluster that can start several instances of mongrail can only create them on one IP with different ports, and pf can’t distribute requests among different ports on same IP.

Here we can create 2 simple scripts that start and destroy server instances:
# cat start.sh

1
2
3
4
5
6
7
8
9
10
count=1
for ip in 10.0.0.1 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5
do
mongrel_rails start -e production -d --user myapp --group myapp --chdir /var/www/railsdocs/myapp \
--address ${ip} \
--pid /var/www/railsdocs/myapp/tmp/pids/mongrel.${count}.pid \
--log /var/www/railsdocs/myapp/log/mongrel.${count}.log

count=$(($count + 1))
done

# cat stop.sh

1
2
3
4
5
for instance in {1..5}
do
mongrel_rails stop --wait 3 --chdir /var/www/railsdocs/myapp \
--pid /var/www/railsdocs/myapp/tmp/pids/mongrel.${instance}.pid
end

And we need to enable pf:
#/etc/rc.conf.local

1
2
pf=YES
#pflogd_flags=                   # add more flags, ie. "-s 256"

add one string to configuration file for load distribution:
#/etc/pf.conf

1
2
rdr pass log on re0 proto tcp from any to SOME_IP port 80 ->
{10.0.0.1, 10.0.0.2, 10.0.0.3, 10.0.0.4, 10.0.0.5 } port 3000 round-robin

check work with:
# pfctl -s nat

monitor with:
tcpdump -n -e -ttt -i pflog0

for more info on pf:
http://www.openbsd.org/faq/pf/
man pf.conf (especially ‘TRANSLATION’ and ‘POOL OPTIONS’ sections)

Then when you see you need more advanced solution for load distribution you can install haproxy.
Check it with [http://www.openbsd.org/4.5_packages/i386/haproxy-1.3.15.7.tgz-long.html]

Twitter directory: our look on how it should work

Monday, July 6th, 2009

“Be the change you want to see in the world.”
Mahatma Gandhi, Indian political and spiritual leader (1869 – 1948)

There are a lot of twitter directories on Internet today. Here I would want to check few most popular ones and provide our look on good directory.

Structure of directory

1. plain list of categories
There is a number of categories on top page, user select one category, and browse all members in that category.

justtweetit
example from: [http://justtweetit.com/]

disadvantages:
a. enormous list of categories on main page.
b. cutting the list of categories creates too broad categories where too many entries are placed.

2. tags for naming of entries
For every entry user can provide number of tags, then users can select appropriate tag.
In theory user has ability to select most narrow meaning and thereby limit number of entries.

wefollow
example from: [http://wefollow.com/twitter/tags/]

disadvantages:
a. there are still too broad tags like ‘celebrity’ with enormous amount of entries.
b. number of tags, so it is hard for user to select most narrow tag.

3. search engine style
Like in search engine user provides search term, and system search for similar entries.

twitdir
example from: [http://twitdir.com/]

disadvantages:
a. user should know twitter name.
And if so user can use build-in twitter search [http://search.twitter.com]. But what if he/she wants to find for example pop style singer? Then user needs directory again!

Administration of directory

All current directories provide possibility to any user to add himself/herself and so suffer from spammers and common(uninterested) users. There should be some quality criteria and some administrators that ensure continues quality of twitter IDs and correctness of provided information.

As it is money consuming tactic directory should make some money to cover costs or use open-source model where authorized users can check entities as it works in DMOZ directory [http://www.dmoz.org/help/become.html]

Will follow Gandhi’s idea

After some discussions within our company we have decided finally to create our own directory of twitter users.
We are taking into account disadvantages of current directories, and so our directory:
*has tree structure to reduce complexity, and size of categories
*uses moderators that ensure quality of twitter IDs before they add them.

Start exploration at: [http://twitwho.net].

Of course there is unlimited ways to improve everything, and so we are open to any suggestions and complains:
helion-prime’s twitter: [http://twitter.com/helionprime]
twitwhohq’s twitter: [http://twitter.com/twitwhohq]

Mozilla Bespin: web-based code editor on web technologies

Friday, February 13th, 2009

Bespin — an experiment from Mozilla Labs that possibly will propose an open, extensible web-based framework for code editing that aims to increase developer productivity, enable compelling user experiences, and promote the use of open standards.

At this time we have initial working experimental prototype that we can use to understand concepts of Bespin and the possibilities that it opens up.
The protorype includes support for basic editing features, such as syntax highlighting, large file sizes, undo/redo, previewing files in the browser, importing/exporting projects.

As for me idea of web-based code editor is arguable due to we have enough excellent standalone open-source code editors like Netbeans [http://www.netbeans.org/] or Eclipse [http://www.eclipse.org/] that have good collaboration possibilities. Standalone code editors we can develop using pretty object-oriented languages like Java and don’t bother with Javascript. But time will say its word.

Mozilla wants you
The Bespin experiment is still in its infancy and just getting started. There are many ways to join the team and get involved: [https://bespin.mozilla.com/]

Firefox 3.0.5: how to fight caching issue

Tuesday, January 20th, 2009

If you are web-developer you probably already heard about regression bug in recent Firefoxes, event latest Firefox 3.0.5 is affected.

Bug is described here on Bugzilla [https://bugzilla.mozilla.org/show_bug.cgi?id=441751], it is fixed, and should be available in new version of Firefox.

As the bug description notes Firefox ignores cache control attributes so we can’t just use them:
Cache-Control: max-age=0, no-store, no-cache, must-revalidate
Expires: Sun, 1 Jan 2000 00:00:00 GMT
Pragma: no-cache

Rare case is you need to tune cache then you can use browser.cache.check_doc_frequency parameter [http://kb.mozillazine.org/Browser.cache.check_doc_frequency] that can be accessed with about:config URL.

But for others question is what to do right now, event with new version of Firefox we can’t ask users to update their Firefoxes.

There are 2 solutions that we have found and tested:

1. If you want to make Firefox to reload some page
You can generate new URLs for any cache sensitive resources every time.
Add some insignificant parameters to URLs: http://www.google.com/something?aaa

2. If you need to make some actions within some page,
for example you need to provide an unique attribute for images in Ads campaign.
You can use Javascript to generate it.
Firefox uses same page but execute Javascript everytime.

You can use following code to generate random number:

1
2
3
4
5
6
7
<script type="text/javascript">
//<![CDATA[

var AdsId = '' + Math.floor(Math.random()*1999999999);

//]]
</script>

I hope men will update their browsers fast so we will not stick with that like with IE6.

RubyConf2008 conference

Tuesday, January 6th, 2009

If you are real Ruby follower you surely know about RubyConf2008 conference that was held in November 2008, or perhaps you even participated. If you still miss all the fun you can go thru whole list of videos from that conference on [http://rubyconf2008.confreaks.com/].

Or at least you can watch videos I’ve selected for busiest men:


name: Reasons behind Ruby
duration: 31 minutes
description: introductory speech from Yukihiro Matsumoto, father of Ruby where he speaks about Ruby, its future, and community in general without any technical stuff.
URL: [http://rubyconf2008.confreaks.com/matzs-keynote.html]


name: Fork Ruby
duration: 48 minutes
description: speech from Dave Thomas that helped write Agile Manifesto, and Programming Ruby: A Pragmatic Programmer’s Guide. He share thoughts about possible ideas for Ruby language and its development.
URL: [http://rubyconf2008.confreaks.com/keynote.html]


name: Ruby 1.9: What to Expect
duration: 50 minutes
description: Dave Thomas and David Black author of popular book Ruby for Rails, Ruby core contributor and the creator and maintainer of RCRchive show to us differences between Rubys with 2 irb windows step by step.
URL: [http://rubyconf2008.confreaks.com/ruby-19-what-to-expect.html]


name: Recovering from Enterprise
duration: 45 minutes
description: Comparison between Ruby and Java worlds from Ruby, and Ruby on Rails contributer Jamis Buck. He describe fundamental differences between Java and Ruby and his mistakes, and ways to avoid them. Also provide ideas how to write in real Ruby and not in Java with Ruby syntax.
URL: [http://rubyconf2008.confreaks.com/recovering-from-enterprise.html]





Next conference will be held on 13-14 March of 2009 in Salt Lake City.
See details on official site: [http://mtnwestrubyconf.org/]


©2010 Helion-Prime Solutions Ltd.
Custom Software Development Agile Company.