Tuesday, July 23, 2013

A Grunting Build

This post is a follow-up on the Dojo and Jasmine entry where we explain how to use the Jasmine framework to unit test a Dojo based application.

In this post, we're going to run our unit tests automatically when the application is built (why the heck would we need to build a javascript app?).

Grunt is described as a JavaScript task runner. It's slowly becoming a defacto-standard in the JavaScript world and it constitutes the central piece of the popular yeoman suite of tools.

To make an analogy with Java build tools, grunt is closer to ant in spirit than maven.

Tasks are defined using a plugin mechanism and an already impressive list is available directly on the site which confirms the popularity of the project. 

Grunt is a node.js  module, so make sure it is setup on your system (remember versions with an even minor number are consider stable, I have 0.10.12). 

Here after my grunt script:

module.exports = function (grunt) {
    var distDir = 'dist/',httpServerPort=9001,httpServerPortTest=9002;
    /*build directory*/
    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),
        //clean up destination dir
        clean: {
            clean: [distDir]
        },
        //minimize javascript modules
        uglify: {
            options: {
                // the banner is inserted at the top of the output
                banner: '/*! <%= pkg.name %> - v<%= pkg.version %> - ' +
                    '<%= grunt.template.today("yyyy-mm-dd") %> */',
                report: 'min'
            },
            dist: {
                files: [
                    {
                        expand: true,     // Enable dynamic expansion.
                        //cwd: '/',      // Src matches are relative to this path.
                        src: ['js/com/**/*.js'], // Actual pattern(s) to match.
                        dest: distDir   // Destination path prefix.
                        /*ext: '.min.js',   // We don't do that on Dojo module because name is important*/
                    }
                ]
            }
        },
        //minimize stylesheets
        cssmin: {
            options: {
                // the banner is inserted at the top of the output
                banner: '/*! <%= pkg.name %> - v<%= pkg.version %> - <%= grunt.template.today("yyyy-mm-dd") %> */',
                report: 'min'
            },
            dist: {
                files: [
                    {
                        expand: true,     // Enable dynamic expansion.
                        cwd: '',      // Src matches are relative to this path.
                        src: ['css/**/*.css'], // Actual pattern(s) to match.
                        dest: distDir   // Destination path prefix.
                    }
                ]
            }
        },
        //static analysis of js modules
        jshint: {
            // configure JSHint (documented at http://www.jshint.com/docs/)
            options: {
                // more options here if you want to override JSHint defaults
                globals: {
                    console: true,
                    module: true
                }
            },
            // define the files to lint
            files: ['js/com/**/*.js', 'test/com/**/*.js']
        },
        //execute a shell command (in that case phantomjs for headless testing)
        exec: {
            //if a local http server is present
            'test-local': {
                command: 'phantomjs test/run-jasmine.js http://localhost:'+httpServerPortTest+'/test/SpecRunner.html'
            },
            //using test machine
            'test-remote': {
                    command: 'phantomjs test/run-jasmine.js http://user:password@192.168.1.100/test/SpecRunner.html'
            }
        },

        copy: {
            main: {
                files: [
                    {expand: true, src: ['text/**', 'img/**', 'js/*.min.js', '*.ico'], dest: distDir}
                ]
            },
            js: {
                files: [
                    {expand: true, src: ['js/**/*.js', '*.ico'], dest: distDir}
                ]
            }
        },
        processhtml: {
            options: {
                data: {
                    version: '<%= pkg.version %>'
                }
            },
            files: {expand: true, src: ['*.html'], dest: distDir}
        },
        connect: {
            options: {
                hostname: "localhost",
                base: "."
            },
            server: {
                options: {
                    port: httpServerPort
                }
            },
            test: {
                options: {
                    port: httpServerPortTest
                }
            }
        }
    });

    // load all grunt tasks
    require('matchdep').filterDev('grunt-*').forEach(grunt.loadNpmTasks);

    // the default task can be run just by typing "grunt" on the command line
    grunt.registerTask('build', ['clean', 'jshint', 'connect:test', 'exec:test-local', 'cssmin', 'copy', 'processhtml']);
    //running grunt dev will run the unit test on the test machine
    grunt.registerTask('remote', ['clean', 'jshint', 'exec:test-remote', 'cssmin', 'copy', 'processhtml']);
    //run this to start up a local http server on current directory
    grunt.registerTask('dev', ['connect:server:keepalive'])

    grunt.registerTask('default', function () {
        var blue = '\033[34m', reset = '\033[0m', green = '\033[32m', cyan = '\033[36m';
        console.log("\n\n");
        console.log(blue + "Welcome to the sample-app grunt build file\n");
        console.log(reset + "The following grunt tasks are available:");
        console.log(reset + "****************************************\n");
        console.log(green + "build:\t\t"+ cyan + "This task builds the entire application. Directory "  + distDir + " will be available for distribution\n");
        console.log(green + "remote:\t\t"+ cyan + "Same thing as build except that tests will be executed remotely against the TEST server. Directory "  + distDir + " will still be available for distribution\n");
        console.log(green + "dev:\t\t"+ cyan + "This will start a local http server listening on port " + httpServerPort + ". This is very handy at development time.");
        console.log("\n\n");
        console.log(reset + "Example:");
        console.log(reset + "********\n");
        console.log(blue + "grunt build");

    });
};

In the first part of the code, we configure the different plugins we're using in our build.

At the end we register some customs coarse grain tasks that simply run some of the tasks we configured earlier.

The default task at the very end will run if no specific argument is given to the grunt command. It simply displays a help instructing the user on how to use the build.

For example running 'grunt build' on the command line cleans the distribution directory, jshint (ref) your js files, starts a local http server, executes unit tests, minifies the style sheets, copies different files and finally processes the html to insert the app version number and replaces development time js to their minified counterpart.

If any of the individual tasks fails, the build is stopped with an error exit code. In order to run our tests, we use the grunt-exec plugin which simply executes a shell command.

PhantomJS is a headless browser (i.e. without a user interface) that is fetching a page from a given url (our Jasmine test runner in this case) and process it just like a regular browser.

PhantomJS is using the Webkit rendering engine also used in Google Chrome (up to version 27) and Safari.

Once the page is loaded we need to instruct PhantomJS what to do with it. In our case we want to know if the Jasmine Test Runner ends up successfully or not. We'd also like to capture the individual tests result and display them in the shell console. The `run-jasmine.js` argument passed on the command line does just that.  This script is available as an example on the PhantomJS site.

Just remember that PhantomJS is not a node module. You will need to setup it up on your build server.

PhantomJS is very handy to run our unit tests but how confident are we that our application will behave correctly with different browsers (IE, FF, Opera)?

That might be the subject of another post. Stay tuned.


Tuesday, June 18, 2013

Dojo and Jasmine

D.O.H. sounding too much like D'oh, I'll then focus on the Jasmine BDD approach, which is both modern and fun to work with.

My folder strcuture:

  • web
    • css
    • img
    • js
      • com
        • acme
          • dashboard
            • Query.js
            • request.js
            • util.js
            • chart
              • fixed.js
            • ...more js modules...

    • less
    • test
      • com
        • acme
          • dashboard
            • Query.js
            • request.js
            • util.js
The Dojo modules I want to test are in the js folder. The index.html page is my production page. The specifications (i.e. test) are in the test folder. In the specs, I completely mirrored the js structure naming the specs identically as the module they're supposed to validate (personal choice).

The SpecRunner.html is the page used to bootstrap the Jasmine Tests.

This is the source:
<head>
    <title>Jasmine Spec Runner</title>

    <link href="lib/jasmine-1.3.1/jasmine.css" rel="stylesheet" type="text/css"></link>
    <script src="lib/jasmine-1.3.1/jasmine.js" type="text/javascript"></script>
    <script src="lib/jasmine-1.3.1/jasmine-html.js" type="text/javascript"></script>

</head>
    <script>
        var dojoConfig = {
            packages: [
                // Any references to a "dash" resource should load modules locally, *not* from CDN
                {
                    name: "dash",
                    location: "/js/com/acme/dashboard"
                },
                // Any references to a "spec" resource should load modules locally, *not* from CDN
                {
                    name: "spec",
                    location: "/test/com/acme/dashboard"
                }
            ],
            asynch: true
        };
    </script>
    <script src="//ajax.googleapis.com/ajax/libs/dojo/1.8.3/dojo/dojo.js"></script>

    <script>
        require(["dojo/ready","spec/util","spec/Query","spec/request"],
                function (ready) {
                    ready(function () {
                        // Set up the HTML reporter - this is responsible for
                        // aggregating the results reported by Jasmine as the
                        // tests and suites are executed.
                        jasmine.getEnv().addReporter(
                                new jasmine.HtmlReporter()
                        );
                        // Run all the loaded test specs.
                        jasmine.getEnv().execute();
                    });
                });
    </script>
In this file I configure the dojo AMD loader to point locally to my modules (in js) and my specs (in test). Dojo itself is taken from a CDN. Any tests I want to run are declared in the require dependencies array. Dojo will call the dependent modules before executing the ready callback preparing all my tests for jasmine to execute.

This is the module under test (js/com/acme/dashboard/request.js) :

define([ "dojo/request", "dojo/json", "dash/constants", "dash/response","dojo/topic","dash/messages", "dash/util"], 
function (request, JSON, constants, response,topic,messages, util) {
    return function (esQuery, type, sync) {
        topic.publish(constants.event_request_in_progress);//in progress handler not working in IE9 and <
        return request.post(constants.es_endpoint, {
            //payload
            data: JSON.stringify(esQuery.getData()),
            handleAs: "json",
            sync:sync
        }).then(
            function (esResponse) {
                var _type = type || constants.query_type_fixed,res = response(esResponse);
                topic.publish(constants.event_response_completed,res,_type);
                return res;
            },
            function (err) {
                topic.publish(constants.event_request_in_error,err);
            },
            function (evt) {
                topic.publish(constants.event_request_in_progress,evt);
            }
        );
    };
});

...And the spec (test/com/acme/dashboard/request.js):

define(["dash/request","dash/Query","dash/constants","dojo/topic"],
    function (request,Query,constants,topic) {
        describe(
            "Testing request to Elastic Search",
            function () {
                var query,es_backup;
                beforeEach(function(){
                    query = new Query();
                    es_backup = constants.es_endpoint;
                    constants.es_endpoint = "http://test_es:9200/classes/student/_search";
                });
                afterEach(function(){
                    constants.es_endpoint = es_backup;
                });

                it("should execute request against Elastic Search on TEST w/o errors", function () {
                    var called;
                    function completed(response,type){
                        called = true;
                    }
                    topic.subscribe(constants.event_response_completed,completed);
                    runs(function(){
                            request(query,constants.query_type_fixed);
                        }
                    );
                    waitsFor(function() {
                        return called;
                    }, "Elastic Search should have been called within 2 seconds on TEST server", 2000);

                    runs(function() {
                        expect(called).toBe(true);
                    });
                });
        );
    });

This spec shows off the interesting Jasmine asynch support. Modules under test are injected into the spec along with any other dojo modules. Next step will be to automate all this. Stay tuned!

Ref http://www.bennadel.com/blog/2393-Writing-My-First-Unit-Tests-With-Jasmine-And-RequireJS.htm


Saturday, November 17, 2012

Securing JEE 6 Applications on JBoss AS7 with an IBM System i LDAP server

I recently had to secure a JEE6 web based application using some credentials stored on a IBM system i (formerly known as AS400). One solution would be to develop a specific JAAS module using the swiss army knife package JTOpen... but with less headache we could achieve the same thing and as a bonus with a more generic solution.

The IBM system i is a complete operating environment that comes with its own LDAP server which we'll leverage for authentication and authorization to our web based application.

I'm using the JXplorer application to manage my server. JXplorer is a LDAP client application that can connect and manage any LDAP v2 or v3 server. The
interface is a little bit clunky (swing baby, swing) but main functionalities are there.

Figure 1 shows a snapshot of the organization I have in my LDAP.

iSeries LDAP Java EE JBoss as 7
Figure 1


For the authentication step, the LDAP server maps the value of the user's 'uid' ldap attribute to the OS400 defined user profile (manageable with the OS command WRKUSRPRF).

The authentication is made directly against the OS so system administrators don't have to manage multiple place for passwords and expiration strategy.

This is a considerable advantage for businesses having an IBM system i as their main box (setting up JBoss AS on AS400 might be the topic of another post).

Users can logon to legacy applications on a 5250 terminal or web applications deployed on JBoss AS using the same set of credentials.


Strategy 1: Assigning Users to Groups


This is the most common scenario where we basically specify which users belong to which groups in copying the user's DN into the designated group's member attribute.

Here after a LDIF showing that strategy:

version: 1
DN: cn=STUDENT,ou=Roles,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
objectClass: organizationalPerson
objectClass: organizationalPerson
objectClass: person
objectClass: groupOfNames
objectClass: top
cn: STUDENT
description: Group of STUDENTS
member: cn=FGARCIA,ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
member: cn=MSTRACHAN,ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
sn: STUDENT

version: 1
DN: cn=FGARCIA,ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: publisher
objectClass: ePerson
cn: FGARCIA
description: Franck Garcia 
mail: franck.garcia@acme.quebec.canada.com
publisherName: dc=as400srv,dc=quebec,dc=canada,dc=acme,dc=com
sn: FGARCIA
uid: FGARCIA



The LDAP groupOfNames object class is the one owning the member attribute so you have to assign it to the STUDENT group on creation (if you're doing it with JXplorer don't forget to use the submit button in order to save your changes).

The attribute can be specify multiple time to record each member of the group.

This is the security domain definition to include in the jboss as 7 conf/standard.xml (or standard-full.xml):

<security-domain name="roster">
    <authentication>
        <login-module code="LdapExtended" flag="required">
            <module-option name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
            <module-option name="java.naming.provider.url" value="ldap://as400srv"/>
            <module-option name="java.naming.security.authentication" value="simple"/>
            <module-option name="bindDN" value="cn=administrator"/>
            <module-option name="bindCredential" value="secret"/>
            <module-option name="baseCtxDN" value="ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM"/>
            <module-option name="baseFilter" value="(uid={0})"/>
            <module-option name="rolesCtxDN" value="ou=Roles,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM"/>
            <module-option name="roleFilter" value="(member={1})"/>
            <module-option name="roleAttributeID" value="cn"/>
        </login-module>
    </authentication>
</security-domain>


We use the com.sun.jndi.ldap.LdapCtxFactory which comes standard in any JVM implementation.


The JBoss LdapExtended module relies on it to communicate with the LDAP server.

The bindDN and bindCredential module options are used to connect and query the LDAP server.

The baseCtxDN and baseFilter options are used by the JBoss LdapExtended module to locate the user attempting to log into the application (the {0} placeholder is replaced at runtime with the user name inputted).

Once JBoss locates the user, it uses the resulting Distinguished Name (DN) to bind again to the LDAP server for authentication.

If authentication succeeds then the roles are assigned to the newly authenticated user.

For this step JBoss uses the rolesCtxDN, roleFilter and roleAttributeID options.

The rolesCtxDN option represents the initial roles search tree, the roleFilter is used to select only the roles the user belongs to (the {1} is expanded
to the user's DN at runtime).

Finally the roleAttributeID option represents the group ldap attribute's value that will be added to the user's roles list.

Strategy 2: Assigning Groups to Users


This strategy is the preferred one of my AS400 administrator... So he made me sweat a bit in order to make it works.

This time the Users owns the LDAP Object class groupOfNames, so we can define the group they belong to directly with the member attribute. If the user belongs to multiple groups, the member attribute can be repeated.

Here after the LDIF:

version: 1
DN: cn=STUDENT,ou=Roles,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
objectClass: organizationalPerson
objectClass: person
objectClass: top
cn: STUDENT
description: Group of STUDENTS
sn: STUDENT

version: 1
DN: cn=FGARCIA,ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: publisher
objectClass: ePerson
objectClass: groupOfNames
cn: FGARCIA
description: Franck Garcia 
mail: franck.garcia@acme.quebec.canada.com
member: cn=STUDENT,ou=Roles,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM
publisherName: dc=as400srv,dc=quebec,dc=canada,dc=acme,dc=com
sn: FGARCIA
uid: FGARCIA


This time we copy the groups DN to the member attribute of the Users instead of the opposite (as depicted in strategy 1).

And here is the jboss security module declaration:

<security-domain name="roster">
    <authentication>
        <login-module code="LdapExtended" flag="required">
            <module-option name="java.naming.factory.initial" value="com.sun.jndi.ldap.LdapCtxFactory"/>
            <module-option name="java.naming.provider.url" value="ldap://dev_caas400"/>
            <module-option name="java.naming.security.authentication" value="simple"/>
            <module-option name="bindDN" value="cn=administrator"/>
            <module-option name="bindCredential" value="secret"/>
            <module-option name="baseCtxDN" value="ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM"/>
            <module-option name="baseFilter" value="(uid={0})"/>
            <module-option name="rolesCtxDN" value="ou=Users,DC=AS400SRV,DC=QUEBEC,DC=CANADA,DC=ACME,DC=COM"/>
            <module-option name="roleFilter" value="(uid={0})"/>
            <module-option name="roleAttributeID" value="member"/>
            <module-option name="roleAttributeIsDN" value="true"/>
            <module-option name="roleNameAttributeID" value="cn"/>                        
        </login-module>
    </authentication>
</security-domain>


The baseCtxDN and baseFilter options are the same ones used in strategy 1.

This time we instruct the JBoss module that the role should be found in the same tree as the Users (by having the rolesCtxDN and roleFilter options equal to the baseCtxDN and baseFilter ones).
The twist is on the roleAttributeID option. We specify that the member attribute of the User LDAP object should be used to identify the roles the user belongs to. But we also informed the JBoss JAAS module that the member attribute just found is a Distinguished Name (by setting the roleAttributeIsDN option to true).
With that knowledge, another LDAP query using this DN is made to locate the corresponding LDAP Group object. Finally the resulting cn LDAP attribute  is selected and added to the user's roles list (roleNameAttributeID).

If you're a bit lost you should follow each steps referring back to the LDIF file.

I also recommend to open the security logging in JBoss AS for troubleshooting:

<logger category="org.jboss.security">
    <level name="ALL"/>
</logger>


This will provide interesting information about authentication and groups attributed to users when they log in.

To finalize you will need to add a specific JBoss DD to your war (ear) in mentioning your newly created JAAS domain :

<jboss-web xmlns="http://www.jboss.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.jboss.com/xml/ns/javaee http://www.jboss.org/j2ee/schema/jboss-web_6_0.xsd"
version="6.0">
    <security-domain>java:/jaas/roster</security-domain>
    <security-role>
        <description>example of mapping an app role name to real group name. This way we can 
        take advantage of already existing groups</description>
        <role-name>STUDENT_ROLE</role-name><!-- the role name appearing in the standard web.xml dd  -->
        <principal-name>STUDENT</principal-name><!-- the existing role in your credential repo -->
    </security-role>
</jboss-web>


An interesting thing to notice is that you could eventually take advantage of already existing groups in your credentials repository and map them to the role defined in the web.xml deployment descriptor.
This is the responsibility of the Application Assembler as described in the java ee specification.

With this indirection the Application Provider might be another person or company selling a product that could be assemble differently at deployment time. 

Here after the standard EE DD web.xml:

<security-constraint>
  <web-resource-collection>
       <web-resource-name>secure home page</web-resource-name>
       <description>intentional login</description>
       <url-pattern>/secure.jsp</url-pattern>
       <http-method>GET</http-method>
       <http-method>POST</http-method>
  </web-resource-collection>
  <auth-constraint>
       <role-name>STUDENT_ROLE</role-name>
  </auth-constraint>
  <user-data-constraint>
       <transport-guarantee>NONE</transport-guarantee>
  </user-data-constraint>
</security-constraint>
<login-config>
  <auth-method>FORM</auth-method>
  <realm-name>roster</realm-name>
  <form-login-config>
    <form-login-page>/login.jsp</form-login-page>
    <form-error-page>/errorPage.jsp</form-error-page>
  </form-login-config>
</login-config>
<security-role>
  <description>All students</description> 
  <role-name>STUDENT_ROLE</role-name>
</security-role>
 
References:

http://www-03.ibm.com/systems/i/index.html
https://community.jboss.org/wiki/LdapExtLoginModule
http://www.openldap.org/lists/openldap-software/200204/msg00756.html
https://docs.jboss.org/author/display/AS7/Security+subsystem+configuration
http://docs.oracle.com/javaee/6/tutorial/doc/bnaca.html
http://www-03.ibm.com/systems/i/software/ldap/
http://jxplorer.org/
http://jt400.sourceforge.net/
http://en.wikipedia.org/wiki/LDAP_Data_Interchange_Format#Tools_that_employ_LDIF