Developing The New Herd

As you may, or may not know, for the last week, i’ve being trying to develop a new version for The Herd – the Ubiquity command aggregator.

I took this decision because the actual version does not work at all – mainly because we’re having too much problems with CouchDB (not that it’s a bad database, but our installation is behaving quite strangely).

The Herd today, is developed on top of Django – and after talking to Atul about this, i decided to drop Django entirely and use web.py.

My main reasons are:

1. Django is a full stack framework that makes you use their templates+their apis+lot’s of things – and for Herd, we wanted to do something like a REST-server with a html+css+js frontend
2. Django doesn’t have a good integration with schema-less databases (we *need* it for herd)

It uses web.py, MongoDB (it’s a great schema-less database) and simplejson for the server and plain html+css+jquery for the front-end (that i really need to improve).

It’s been 9 days now, and the new version is quite usable – it already fetches/parsers the feeds.
You can login (using openid) and comment on feeds (voting is almost complete too).

I hope i can “finish” it by the end of the week so i can really focus on the ui. (i will need some help on that).

Some screenshots:
The New Herd

The New Herd

Posted in ubiquity | Leave a comment

Using Oracle Coherence with Grails Part I

Where i work, each with we’re assigned to a different tasks. My tasks this week are: test Beehive Webservices (i’ll post about it later) and test Oracle Coherence and Terracota.

I decided to start with Coherence because it really called my attention — it’s a in-memory jcache compliant clustered cache.

I started reading some docs about it and discovered that it can act as an 2nd-level-cache to Hibernate, so why not give a try with grails?

Everything is quite easy:

First, you have to download coherence zip to your machine and extract it. You will have a dir structure like this:


coherence
|– bin
|– doc
|– examples
`– lib

To start an Coherence console, just type bin/coherence.sh (with your JAVA_HOME set). It’ll show a lot of output, but the main thing is:

Services
(
TcpRing{TcpSocketAccepter{State=STATE_OPEN, ServerSocket=192.168.0.2:8088}, Connections=[]}
ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.3, OldestMemberId=1}
)

Here it shows the cluster nodes -- Coherence will always search for nodes on your machine through network broadcasting. So, if you start another console (on another machine, or on the same, you decide) you'll see:

Services
(
TcpRing{TcpSocketAccepter{State=STATE_OPEN, ServerSocket=192.168.0.2:8089}, Connections=[]}
ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.3, OldestMemberId=1}
)

Map (?):
2008-08-25 21:18:10.026 Oracle Coherence GE 3.3.1/389 (thread=TcpRingListener, member=2): TcpRing: connecting to member 1 using TcpSocket{State=STATE_OPEN, Socket=Socket[addr=/192.168.0.2,port=50602,localport=8089]}

Here Coherence shows that it recognized the other node you started. (BTW on the other console you'll see a message like this too)

If you type who on any console, coherence will tell you how many machines are on the cluster and who are them:

MasterMemberSet
(
ThisMember=Member(Id=2, Timestamp=2008-08-25 21:18:08.241, Address=192.168.0.2:8089, MachineId=26626, Location=process:30900@rome)
OldestMember=Member(Id=1, Timestamp=2008-08-25 21:16:35.527, Address=192.168.0.2:8088, MachineId=26626, Location=process:30378@rome)
ActualMemberSet=MemberSet(Size=2, BitSetCount=2
Member(Id=1, Timestamp=2008-08-25 21:16:35.527, Address=192.168.0.2:8088, MachineId=26626, Location=process:30378@rome)
Member(Id=2, Timestamp=2008-08-25 21:18:08.241, Address=192.168.0.2:8089, MachineId=26626, Location=process:30900@rome)
)

Now let's start with the funny part (playing w/ Grails!)

First, you have to put coherence jars on your app -- coherence.jar, tangosol.jar and coherence-hibernate.jar -- this last one being the one with the 2nd level cache classes.

Then, put on your Datasource.groovy:


hibernate {
cache.use_second_level_cache=true
cache.use_query_cache=true
cache.provider_class='com.tangosol.coherence.hibernate.CoherenceCacheProvider'
show_sql = true
}

The bold part is the main thing -- is the hibernate's 2nd level cache implementation. I recommend you to put the show_sql to true, just to see when a query hits the database.

Then create a file called hibernate-cache-config.xml (ok, the actual name doesn't really matter...) and put the following content on it: http://pastebin.com/f7c883504
This file contains the configuration on how the cache should be created, replicated and etc.

Now, put

static mapping = {
cache true
}

on the domain classes you want to cache. (this is important -- and i didn't know that, but thanks to Les Hazlewood's explanation on this thread i know why i need it)

Now, let's test it!

Fire-up a grails console instance with the following command:

grails -Dtangosol.coherence.cacheconfig=./hibernate-cache-config.xml -Dtangosol.coherence.hibernate.cacheconfig=./hibernate-cache-config.xml console

With this command you set two system variables (coherence uses them) that point to the file we created later.

After typing enter (duh!) you will see a lot of messages about coherence (just like the ones you saw when starting the console) (note: i recommend you not to have any other instance of coherence running when doing these tests). When the coherence finally started, save some instances of your domain class(ess) -- i.e.:

import com.tangosol.net.CacheFactory

(1..25).each { new Author(firstName:"${it}", lastName:"${it}", birthday:new Date()-1).save() }

Then run this code on the same console (assuming your domain class is called Author):


import com.tangosol.net.CacheFactory

cache = CacheFactory.getCache(Author.class.name)

Author.get(1)
Author.list()

println "Cache size: ${cache.size()}"
println " ---- Cache Content ---- "
cache.each {k,v ->
println "Key: ${k} -- Value:${v}"
}

return null

On my machine, i see this output:


Hibernate: select author0_.id as id1_0_, author0_.version as version1_0_, author0_.birthday as birthday1_0_, author0_.first_name as first4_1_0_, author0_.last_name as last5_1_0_ from author author0_ where author0_.id=?
Hibernate: select this_.id as id1_0_, this_.version as version1_0_, this_.birthday as birthday1_0_, this_.first_name as first4_1_0_, this_.last_name as last5_1_0_ from author this_
Cache size: 25
---- Cache Content ----
Key: Author#1 -- Value:Item{version=0,freshTimestamp=1219712346919
Key: Author#2 -- Value:Item{version=0,freshTimestamp=1219712346928
Key: Author#3 -- Value:Item{version=0,freshTimestamp=1219712346929
Key: Author#4 -- Value:Item{version=0,freshTimestamp=1219712346930
Key: Author#5 -- Value:Item{version=0,freshTimestamp=1219712346931
Key: Author#6 -- Value:Item{version=0,freshTimestamp=1219712346933
Key: Author#7 -- Value:Item{version=0,freshTimestamp=1219712346934
Key: Author#8 -- Value:Item{version=0,freshTimestamp=1219712346935
Key: Author#9 -- Value:Item{version=0,freshTimestamp=1219712346936
Key: Author#10 -- Value:Item{version=0,freshTimestamp=1219712346938
Key: Author#11 -- Value:Item{version=0,freshTimestamp=1219712346939
Key: Author#12 -- Value:Item{version=0,freshTimestamp=1219712346944
Key: Author#13 -- Value:Item{version=0,freshTimestamp=1219712346945
Key: Author#14 -- Value:Item{version=0,freshTimestamp=1219712346947
Key: Author#15 -- Value:Item{version=0,freshTimestamp=1219712346948
Key: Author#16 -- Value:Item{version=0,freshTimestamp=1219712346949
Key: Author#17 -- Value:Item{version=0,freshTimestamp=1219712346951
Key: Author#18 -- Value:Item{version=0,freshTimestamp=1219712346952
Key: Author#19 -- Value:Item{version=0,freshTimestamp=1219712346955
Key: Author#20 -- Value:Item{version=0,freshTimestamp=1219712346956
Key: Author#21 -- Value:Item{version=0,freshTimestamp=1219712346957
Key: Author#22 -- Value:Item{version=0,freshTimestamp=1219712346960
Key: Author#23 -- Value:Item{version=0,freshTimestamp=1219712346961
Key: Author#24 -- Value:Item{version=0,freshTimestamp=1219712346962
Key: Author#25 -- Value:Item{version=0,freshTimestamp=1219712346963

What this means is that we have all 25 objects on cache! Uhuu!

On the next post....
- Add this to a real web-app
- Create a web interface to show what objects we have on cache and clean it

See you next time.

Posted in groovy, technology | Tagged , , , , | 4 Comments

Auto Increment On A Oracle DB

This week i have a task: explore JHeadstart — Easy — specially if you follow their (uncommented) tutorial. But, when you start to build your own app, things start to get a little confusing (and, difficult) — specially because, by default, the framework creates screens with ids (the user can input whichever id they want) and this is not the right thing on a real-world-app. Ok, but if the user is not going to input the id, we have to create an auto-increment on the column.

Easy. Just put the auto_increment on the column…. No, not.
On Oracle DBs, different of MySQL, you cannot set an column to auto_increment, and it’s not easy to discover how to do it (if you don’t have an oracle dba in the house).

So, god google showed me a link telling me how to do it. If you don’t want to click on the link, i’ll show it for you here.

First, you need to create an sequence — the sequence is the ‘auto-increment-thing’, the one that’ll give to you the numbers.


create sequence funcs_id_seq
start with 1
increment by 1
nomaxvalue;

Then, you will have to create a trigger, that'll be triggered when you insert an new row into the table:


create trigger func_id_trigger
before insert on functionaries
for each row
begin
select funcs_id_seq.nextval into :new.id from dual;
end;
/

Then, just to an normal insert into your table (w/o the id, of course) and the id will be setted!

Posted in technology | Tagged , , | Leave a comment

Grails Auto Complete

Yesterday i finished my script for command completion on grails. It’s quite complete right now (not perfect, but almost complete).
I thought it would be more difficult to do it, but thanks to tutorials (here and here) and the other script for completion, it was quite easy.

The whole script (along with the installation and how-to-use-it can be found here: http://www.grails.org/Grails+Bash+Completion+Script

Posted in groovy, technology | Tagged , , , , , | 2 Comments

Ajude a sustentar a Wikipédia e outros projetos, sem colocar a mão no bolso, e concorra a um Eee PC!

…e também a pen drives, card drives, camisetas geeks, livros e mais! O BR-Linux e o Efetividade lançaram uma campanha para ajudar a Wikimedia Foundation e outros mantenedores de projetos que usamos no dia-a-dia on-line. Se você puder doar diretamente, ou contribuir de outra forma, são sempre melhores opções. Mas se não puder, veja as regras da promoção e participe – quanto mais divulgação, maior será a doação do BR-Linux e do Efetividade, e você ainda concorre a diversos brindes!

Posted in technology | Tagged , | Leave a comment

Adding Autocomplete to a Python Console

I really love python. It’s a great language (fyi it’s one of the languages that google uses — they are java, python, c and c++) and has a powerful console.
But, it misses an great point: autocomplete. (A think that ruby’s console has)

So, searching on the internet i found this site that show how to do it:

First, create a new environment variable called PYTHONSTARTUP pointing to a file “$HOME/.pystartup” — you can do that using the command export PYTHONSTARTUP=”$HOME/.pystartup. (You can add the command to your .bashrc, so you don’t need to type everytime you startup a console)

Then you create the file .pystartup and add the same content to it:

# Add auto-completion and a stored history file of commands to your Python
# interactive interpreter. Requires Python 2.0+, readline. Autocomplete is
# bound to the Esc key by default (you can change it - see readline docs).
#
# Store the file in ~/.pystartup, and set an environment variable to point
# to it, e.g. "export PYTHONSTARTUP=/max/home/itamar/.pystartup" in bash.
#
# Note that PYTHONSTARTUP does *not* expand "~", so you have to put in the
# full path to your home directory.

import atexit
import os
import readline
import rlcompleter

historyPath = os.path.expanduser("~/.pyhistory")
historyTmp = os.path.expanduser("~/.pyhisttmp.py")

endMarkerStr= "# # # histDUMP # # #"

saveMacro= "import readline; readline.write_history_file('"+historyTmp+"'); \
    print '####>>>>>>>>>>'; print ''.join(filter(lambda lineP: \
    not lineP.strip().endswith('"+endMarkerStr+"'),  \
    open('"+historyTmp+"').readlines())[:])+'####<<<<<<<<<<'"+endMarkerStr

readline.parse_and_bind('tab: complete')
readline.parse_and_bind('\C-w: "'+saveMacro+'"')

def save_history(historyPath=historyPath, endMarkerStr=endMarkerStr):
    import readline
    readline.write_history_file(historyPath)
    # Now filter out those line containing the saveMacro
    lines= filter(lambda lineP, endMarkerStr=endMarkerStr:
                      not lineP.strip().endswith(endMarkerStr), open(historyPath).readlines())
    open(historyPath, 'w+').write(''.join(lines))

if os.path.exists(historyPath):
    readline.read_history_file(maxhistoryPath)

atexit.register(save_history)

del os, atexit, readline, rlcompleter, save_history, historyPath
del historyTmp, endMarkerStr, saveMacro

Now you can start your python console typing python (Duh!) and try to do some autocomplete like:
a = "Fernando"
a.
a.is
a.islo

Enjoy!

Posted in technology | Tagged , , , | 5 Comments

Why i like Groovy

This is why i like groovy. I follow an brazilian java list, and last week, a guy posted some logic exercices in java. One of them was:

Given an array of numbers, create a function that searches all instances of a given number an print the number of occurences of this number.
Another guy posted the answer in java:

public class Modulo{
public static  int EncontraNumero(double[]  vetor, double  x, int count){
for (int i = 0; i < vetor.length; i++) {
if (vetor[i]== x)
count++;//calcula quantas vezes apareceu no vetor

}
return count;
}
public static void main(String[] args) {

double[] vet={2.3,5.7,9.65,3.33,4.10,5.23, 12.5,  12.5};
System.out.println(EncontraNumero(vet, 12.5, 0));
}
}

I created the same thing with groovy:


numbers = [2.3,5.7,9.65,3.33,4.10,5.23, 12.5, 12.5]
println numbers.findAll {it == 12.5 }.size

15 lines in 2. Wonderful

Posted in groovy, technology | Tagged , , | 3 Comments