Friday, December 19, 2014

Groovy, Gorm with Spring XD


This piece covers the setup and use of Gorm (Grails ORM layer) as a stand-alone entity that can be used by Spring XD jobs and modules.


Foremost - why? What's Gorm and why use it with Spring XD?


Gorm is an extension to Hibernate in a way. Hibernate is good at ORM i.e. mapping classes to database objects. Hibernate is also good at querying. What Hibernate is not good at is the actual usage. There's simple too much you have to do get things going and too much noise afterwards. You have to annotate classes (or worse use XML descriptors), you have to setup persistent context, you have to use Hibernate session to execute actual queries. Now these are the areas addressed by Gorm. With Gorm you can use meaningful defaults. Let's have a look at this Groovy class

class EntityName {
String name
String value
String someOtherValue
Date lastUpdated
Date dateCreated
}

A class like this when seen by Gorm would automagically get converted into something similar

@Entity 
@Table (name = "entity_name")
class EntityName {
@Id @GeneratedValue
    @Column(name = "id")
int id

@Version
int version

    @Column(name = "name")
    String name
    
    @Column (name = "value")
    String value
    
    @Column (name = "some_other_value")
    String someOtherValue

    @Column (name = "last_update")
Date lastUpdated

@Column (name = "date_created")
Date dateCreated

    
    @PrePersist
    void onCreate() {
       dateCreated = new Date();
    }

    @PreUpdate
    protected void onUpdate() {
       lastUpdated = new Date();
    }
}

As you can see you can get a lot for free just by using Gorm. Now that's just a tip of the iceberg as with Gorm you also get validations, constraints and many, many more.

Hope it's clear now that it makes sense to use Gorm even outside of Grails. So what else you have to do to get Gorm running in Spring XD?

First of all, you have to get all Gorm dependency classes loaded by the initial classloader. To this end you can either copy all the classes into the lib folder (you don't really want to do that do you?) or tweak the startup script a bit and add another folder , say called ext, to added onto the classpath. See the attached patch for details.

Now with all the required classes on the classpath we can start playing with Gorm. First, we need to add it into our module's context.

<gorm:sessionFactory base-package="your.package" data-source-ref="dataSource" message-source-ref="messageSource">
    <property name="hibernateProperties">
     <util:map>
                <entry key="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect"/>
        </util:map>
    </property> 
</gorm:sessionFactory>        

Obviously you have to define the messageSource and dataSource (dataSource is normally already bound the one define in the main yaml file) and the messageSource is easy to define

<bean id="messageSource"
          class="org.springframework.context.support.ResourceBundleMessageSource">
    <property name="basename" value="messages"/>
</bean>

Just add all your messages files on the classpath (your module config will suffice). And that's pretty much it.

All your domain classes need to be annotated by the import grails.persistence.Entity`annotation and have to implement equals and hash - for the lazy use the import groovy.transform.EqualsAndHashCode annotation and an AST transformation will take care of it for you.

There are several things you have to keep in mind should you wish to share the model classes between stand-alone Gorm and Grails' Gorm.


  • Autostamping is not working (dateCreated, lastUpdated). You can always use the beforeUpdate closures though.
  • You have to use session explicitly (DomainClass.withSession) or make a component using Gorms domain classes org.springframework.transaction.annotation.Transactional. In both cases you just make sure the Hibernate session is threadlocal.
  • The Entity annotation adds (or more specifically AST adds) some extra fields to the domain classes. Gorm then tries to blindly maps those to the underlaying model and the fails. To avoid this add these fields into the transient section


static transients = ['errors', 'instanceGormInstanceApi', 'instanceGormValidationApi', 'staticGormStaticApi', 'instanceConvertersApi', 'attached', 'dirty'] 

The source can be found on Github.

Wednesday, November 12, 2014

Integrating Grails 2.4.4 with CMS

In our company we use Typo3 CMS to maintain our dynamic contents - copywriting, descriptions etc. The content is actually stored as GSP pages - meaning even in our CMS we use tags such as <a:message/>or our custom tags <z:whatever/>. Basically the way it works we go to CMS every time our URLMapping matches certain pattern to extract and display the content. Once we retrieve the information we use the GroovyPagesTemplateEngine, as provided by Grails, to render the page. Now, with this rendered content we just call the render method to get the actual content rendered providing the layout parameter so the content nicely fits into our app. And it works surprisingly. In a bit of pseudo-code it looks like this:


def template = HTTPClient.get (cmsURL)
def template = groovyPagesTemplateEngine.createTemplate(template, "whateverName")
def result = new StringWriter()
template.make().writeTo(result)
def pageContent = result.toString()
render(text:pageContent, encoding:"UTF-8", type:"text/html", layout: layout)

As you can see it is a fairly straight forward solution. No catches here. Obviously under the hood we do pretty crazy stuff as Grails has to parse the template and execute it in the context of the current request, however from our point of view it looks simple.

All this worked great on Grails 2.3. Then we decided to move onto Grails 2.4. We upgraded a couple of dependancies and so on and were looking forward to see our pages in the new shiny Grails 2.4 look and feel. That didn't happen though. The reason was our CMS. Pages were fetched, templates rendered however the output was stripped, we could only see blank pages and we were asking WHY!?

After several hours of looking under the bonnet of our application we'd finally figured out the reason. To actually explain the why let me just briefly explain how layouts work in Grails 2.3 and Grails 2.4. In version 2.3 Grails uses Sitemesh filter. Basically all layout related operations are stored in a request object and once they leave controllers they're processed by the Sitemesh filter. From this point of view it doesn't matter at what stage you do what with layouts - all is post-processed in the filter. Grails 2.4 has ditched the filter and applies Sitemesh in the render method. Although it looks like a minor change it actually means a lot. It most importantly means that timing is important now. If you just render your pages then the impact is zero - if you compile your GSPs on the fly you have to do several steps to make the process working.

First you have to prepare the request for the initial GSP rendering. That would normally happen during the rendering but we run our GSP before we call render so we have to do it manually. Now when rendering templates all layout elements are processed by the RenderTagLib. This library finds all layout related tags and saves them into a GSPSitemeshPage instance stored in a request. So what we do now is that before calling the initial rendering of our template coming from CMS we create an instance of that class and store it in a request. Once this is done we move that particular instance into a different place in request where the layout rendering can locate it and we do the layout rendering. After that we have our HTML nicely stored in a String variable. The last step is to render it as a text. In a pseudo code it could look like:


static final GSP_SITEMESH_PAGE ='org.codehaus.groovy.grails.web.sitemesh.GrailsLayoutView.GSP_SITEMESH_PAGE'
static final PAGE = "__sitemesh__page";


def template = HTTPClient.get (cmsURL)
request.setAttribute(GSP_SITEMESH_PAGE, request.getAttribute(GSP_SITEMESH_PAGE)?: new GSPSitemeshPage(false)) // that's where we prepare the empty sitemesh page
def sitemeshPage = request.getAttribute(GSP_SITEMESH_PAGE)
def template = groovyPagesTemplateEngine.createTemplate(template, "whateverName")
def result = new StringWriter()
template.make().writeTo(result)
def pageContent = result.toString()
def template = (GroovyPageTemplate) findLayout(request, layoutName)
result = new StringWriter()
request.setAttribute(PAGE, page) //layout rendering needs the simemesh page here
template.make()writeTo(result)
pageContent = result.toString()

render(text:pageContent, encoding:"UTF-8", type:"text/html")


And that's the end of the story. Just keep in mind that every GSP template you load from CMS is actually compiled into Java class and loaded by the classloader. That means that naming the templates is more important than you might have actually thought!

Tuesday, August 12, 2014

The rise and fall of ORM - object relational mapping

You know the story - you start programming and you find yourself in need of a data store - place to keep and lately, hopefully, retrieve your precious data. You already know some Java, HTML, CSS, probably JavaScript as well and now you here comes the database. You've got several options so you start choosing - NoSQL, SQL or something else. Modern developers pick NoSQL for sure, they just want to save data and after us, the deluge. I'm going to skip it as I truly belief that NoSQL will eventually become a marginalized minority as it was several years back.
When it comes to SQL you start to have a dilemma. You'd like to access the database in an object way however there's this hairy old SQL language which doesn't do objects. So your first instinctive thought is to start using ORM - and nowadays Hibernate is a synonym for ORM.
It goes all so well. You define your data objects, you let Hibernate to generate your database schema - happy days. You can query for objects, create, modify and delete them and it's all so rosy.
But in a while - problems. Transactions not working as expected, relations are not propagated the way you want them to, database becomes slow and keeping the application running a chore. And after a bit longer you start to be more and more unhappy with your choice.
So what's wrong with ORM?
First of all ORM adds an extra level of complexity. You have to define stuff and obviously ORM brings in additional extra megabytes of code. And this comes at price. Application start up time up, performance down, scope for errors even higher. Hibernate uses black magic to get lots of things done so you find your object and method proxied, AOPed and intercepted. And that's not all.
ORM encourages bad practices and accesses database in an uncontrolled way - instead you taking control of the database you may not even know what tables there are!
Later you figure out that optimizing queries is next to impossible. Queries are generated by the ORM framework and you've got only limited way to influence their creation. So things like adding an index can be a nightmare.
And that's not all - ORM gets you more data than you need or you think you get. Not only you can’t specify columns you want to get, you may also find that you're fancy ORM goes to database for every detail object. I'm talking about this typical scenario where you iterate a collection of details. You either get all eagerly or nothing lazily. But ORM usually has no idea how you want to get the data (and neither do you) so if a detail for a master has 1000 elements you can end up going to database 1000 times. A real performance killer.
And even if you try to optimize that thing you may struggle - ORM makes it hard to optimize DB by adding indexes due to its unpredictability when it comes to construction of SQL queries. You can, normally, dump all the queries but there's no guarantee that the same queries will be generated all the time so it may be difficult to figure what to index and how. It's also a nightmare for your DBAs as generated queries are hard to read and thus optimizing can be a good fun for a couple of weeks.

So it's the old fashioned DAO where your handcrafted SQL lives still a way to go? May be. That's mostly up to you. But either way, you better master those SQL skills as that's the only way to get your database performance where you want it to be.

Friday, June 27, 2014

Is AngularJS the new Fat client?

It was year 2000. People survived Y2k, the dot com bubble was just about to burst and we were writing our shiny applications. Many people were already doing HTML based thin clients (with a bit Javascript and mostly on Internet Explorer because Netscape was crap with anything beyond basic form validation).
But at that time there were still people doing something called "Fat client".

So what exactly was (is) a Fat client application.

In a nutshell, fat client is a concept where the client holds the logic of the application and talks to database directly. Database in this case is the point of truth. Fat clients have certain advantages.

  • Fat client utilises the users CPU and resources - thus spares the company from buying (at that time) expensive servers.
  • Fat client is pretty quick - unless you need something from the database (backend) and the database is busy.
  • Fat client's look and feel is consistent. It is a native application so users are usually familiar with the UI.


But there are certain drawbacks

  • Fat client needs to be installed and maintained locally. It's OK for several users, but thousands users can be a completely different story.
  • Fat client runs on one platform only - it is a native application in the end! It means that upgrade to newer version Windows is risky not even thinking about porting that application to Unix/MacOS.
  • Fat client can simply die or stop working and you don't know it until your users tell you (if they even bother to tell you).

And the list goes on.

Now, fast forward, year 2014. Here we are. We've got our servers but we try again to let our clients to do the heavy lifting. This time with a small twist. Instead of using native applications (we've been there, haven't we?) we use HTML/Javascript. So what exactly differs between the fat client v.1 (native application in year 2000) and fat client v. 2 (AngularJS application in year 2014)

  • We dropped native applications, we now use HTML and Javascript. So the UI may be completely unfamiliar to our users. HTML builds a very thick layer on top of the hardware so our fat client v. 2 applications may be pretty slow on older computers.
  • We ditch the installation process. Installing software is bad. Updating, maintaining it - no kudos there. Instead, we install the application over and over again as the user visits our site. But is the user really visiting our site that often? What the users just put his PC on stand by and have the page (application) running 24/7 for a days, months?
  • We're smarter now - we don't talk to database. We talk to APIs. And because we're smarter we all use JSON. JSON is fine, everyone can read it, it's nicely structured and all the network sniffers are happy - listen or port 80 (443) and all the structured data is there. 
  • As the application is written in Javascript everyone can read the source code and tweak it. All these wannabe developers rejoice.
  • We are more masochistic now. If something looks easy then everyone could do it, sure thing. So we make Angular difficult. Controllers, services, directives, dependancy injection. Who cares you only want to let user to input several parameters to get some output back, you still need to make it painful for the developers, they need to deserve their pay.
  • We learn from experience and we know users hate error messages. That's why we don't give them any. If we're in a good mood we dump something on the console and we pray for the user not to press F12 (or CMD+ALT+J). Ideally we let to user in the dark by presenting a screen where things look working but the code behind them is long time zombie.
  • Unfortunately the browser still has the reload and back button. We just hope the user will not press it or we try to come up with a smart way handling that. Users are confused but who cares about their feelings. 
  • Using browser has other nice advantages. No native printing, no access to the hardware, not working app if the user happen to lose his Internet connectivity. Some big companies allow you to continue using a crippled version of their application if you're offline but most people don't bother.
  • And we all know that data is the king. So we don't allow the user to access the data. Fat client v. 1 would allow you to save data in various formats but v. 2 saves data in the Cloud and makes sure anything that even slightly resembles the save button is removed.


So, is it all lost? I don't think so. Because there's already Fat client v. 3 here. Oh yes - the native smartphone/tablet applications! They install but keep themselves updated, they talk to APIs but in their own formats (although JSON is getting there as well). The applications still crash, allow you to save data and work offline. And now (*drumroll*) Google would like to allow these apps on their Chromebooks - eat that Angular!

And for those who want to see this side by side - here's the link!