Aliasing Loopback on OSX
aka…how to make it possible to use 127.0.0.2, 127.0.0.3, … on your Mac.
This post comes from a need I had to start cassandra in multiple instances on a single Mac (testing a cluster configuration out), and failing to do so. Previously on Linux, I had configured multiple instances to bind to 127.0.0.1 and 127.0.0.2 and it worked without any trouble. On the Mac though, I got the exception:
java.net.BindException: Can't assign requested address at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52) at org.apache.cassandra.net.MessagingService.listen(MessagingService.java:138) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:319) at org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:99) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
Until Jonathan Ellis and Peter Schuller on the cassandra list told me I could alias the extra addresses to loopback using this:
sudo ifconfig lo0 add 127.0.0.2
After that, ifconfig shows the added IP under the “lo0″ device and cassandra can connect to it.
July 26, 2010
Tags: cassandra, mac, OSX Posted in: Operating Systems 
No Comments
A Non-Blocking Process.waitFor()
When calling java.lang.Process.waitFor() the call will block (wait()) until the process actually exits. The waitFor() call is blocking, which is pretty irritating sometimes, for instance when #&%€! wine decides to hang itself, hereby stealing my tomcat execute threads and database connections in production 
Well, here’s a quick and dirty solution to creating a non-blocking waitFor() call:
public boolean nonBlockingWaitFor(Process proc, long procMaxTimeoutMillis) throws InterruptedException {
final long TIMESTAMP_BEFORE_EXECUTE = System.currentTimeMillis();
boolean hasExited = false;
while (!hasExited) {
try {
// set/calculate your own granularity of check, which fits your procMaxTimeoutMillis
Thread.sleep(500);
// using the IllegalThreadStateException side-effect of exitValue() if process hasn't exited
proc.exitValue();
hasExited = true;
} catch (IllegalThreadStateException e) {
if (System.currentTimeMillis() > (TIMESTAMP_BEFORE_EXECUTE + procMaxTimeoutMillis)) {
break;
}
}
}
return hasExited;
}
This little block of code makes use of the fact, that Process.exitValue() will throw IllegalThreadStateException if one tries to peek at the exit value before the process has exited.
Is it nice? No!
Does it work? Yes!
July 16, 2010
Tags: Java Posted in: Programming No Comments
Testing CXF with Autowiring using Spring
Here is how to create a test for a CXF web service implementation class. The test use spring-mock base classes and hereby support stuff like transactions and autowiring through annotations. Neat.
I have a base class like this:
import org.springframework.test.jpa.AbstractJpaTests;
public abstract class AbstractCxfWsBeanTest extends AbstractJpaTests {
public static final String[] SERVICE_CONFIG_LOCATION = new String[]{
"classpath*:/applicationContext.xml",
"classpath:/datasourceContext.xml",
"classpath:/cxf.xml",
};
@Override
protected String[] getConfigLocations() {
return SERVICE_CONFIG_LOCATION;
}
}
One thing here: cxf.xml is loaded from classpath by the test code, but it resides in src/main/webapp/WEB-INF in my source tree, which is not exactly classpath stuff. I came around that with this little snippet in the <build> section of the maven pom.xml:
<build>
<testResources>
<testResource>
<directory>src/test/resources</directory>
<includes>
<include>*</include>
</includes>
</testResource>
<testResource>
<directory>src/main/webapp/WEB-INF</directory>
<includes>
<include>cxf.xml</include> <!-- needed to make tests have cxf beans injected on them -->
</includes>
</testResource>
</testResources>
</build>
And finally, I can write a test like this one:
import org.springframework.beans.factory.annotation.Autowired;
public class TestMyWebService extends AbstractCxfWsBeanTest {
@Autowired
private MyWebService myWebService; // this is where I get my CXF WS impl injected
public void testFoo() {
assertEquals("bar", myWebService.foo());
}
}
June 20, 2010
Tags: cxf, Java, maven, spring, test Posted in: Programming No Comments
One Pool to Rule Them All…in tomcat
I am developing a system that gets deployed as a bunch of separate webapps but all in the same tomcat instance. Not long ago, we opened up production for a lot more users, which quickly led to some resource exhaustion-primarily on the database backend. In other words, … we were beating oracle to death.
After some investigation we concluded, that, …the database was too loaded. Duh! Okay, so why? It turns out we hadn’t been using the insides of our heads that much. Each webapp deployed on tomcat had its own database pool defined in its own context. With many webapps, this summed up to a large max-value for how many connections we could open against oracle. Something that oracle couldn’t scale to.
So, to maximize throughput and sacrificing a bit on individual request processing time, we decided to define one single pool in tomcat, and let each webapp reference that pool. Here’s how it’s done:
In conf/server.xml in the <GlobalNamingResources> element, define the global database pool with a <Resource> element like we normally do inside each context. Something like this:
<GlobalNamingResources>
<Resource name="jdbc/yourglobaldsname" auth="Container" type="javax.sql.DataSource"
maxActive="20" maxIdle="5" maxWait="20000"
username="scott" password="tiger" driverClassName="oracle.jdbc.OracleDriver"
url="jdbc:oracle:thin:@dbhostname:1521:sidname" initialSize="0" minIdle="1"
validationQuery="select count(*) from smalltable"
testOnReturn="false" testOnBorrow="true" timeBetweenEvictionRunsMillis="60000"
numTestsPerEvictionRun="20" minEvictableIdleTimeMillis="120000"/>
</GlobalNamingResources>
Then, in each context you remove the resource definition of the per context pool, and instead inserts a <ResourceLink> element, to refence the global pool. Like this in conf/Catalina/localhost/yourcontext.xml:
<ResourceLink name="jdbc/yourdsname" global="jdbc/yourglobalds" type="javax.sql.DataSource" />
With a lot less load on the database, all users get a chance to get their work done. So by actually lowering the amount of concurrent requests we allow against the database, we get both better throughput and better processing time pr. request.
June 13, 2010
Tags: deployment, Java, pool, tomcat Posted in: Programming No Comments
@Grab Support in IntelliJ IDEA
Recently I wrote about the great Grape stuff in Groovy for use when deploying groovy scripts.
Just wanted to give a quick view on what IDEA can do for you with some of the latest JetGroovy support for @Grab annotations. With my projects I use maven for building, and as such, I have my dependencies in the pom. I then use the excellent maven integration in IDEA by simply opening project from the pom.xml, hereby letting IDEA build dependencies for all modules. Works well.
With my groovy code module though, I have the dependencies listed as @Grab annotations instead, as that will help my deployment. What I do not want then, is to have to put them into the pom.xml too. In essence, IDEA should be able to add module dependencies by recoqnizing the @Grab annotations.
Luckily, this is under way, and you can already now try it out. There is a small extra Grab plugin, that needs to be installed. When you have that plugin installed, you get an intention action for the @Grab annotations, saying if you would like to download. If executed, they will also appear on the module dependencies, nicely marked with “Grab: ..” names.
For further info see this issue and the Grab plugin can be found here.
One small note: If running maven-based projects in IDEA like me, you might need this Ivy dependency
<dependency>
<groupId>org.apache.ivy</groupId>
<artifactId>ivy</artifactId>
<version>2.1.0</version>
</dependency>
…in your pom to make the plugin work. I think a fix is under way for this though 
April 21, 2010
Tags: grab, grape, groovy, IDEA, plugin Posted in: Programming, Tools No Comments
Groovy Grape – Adding Dependencies to Root ClassLoader
When using grape in groovy to download dependencies and add to classpath, they will by default be added to the GroovyClassLoader when the script is run. This can be a problem, if these dependencies needs to be uses from classes higher in the classloader hierarchy. One common example would be when loading JDBC drivers, because these are loaded using Class.forName. So, this code will fail with ClassNotFoundException:
@Grapes([
@Grab(group = 'oracle', module = 'ojdbc15', version = '11.2.0.1.0'),
])
class SqlTool {
private sql;
public SqlTool(url, driver, username, password) {
this.sql = Sql.newInstance(url, username, password, driver)
// ... do sql stuff
}
}
new SqlTool("jdbc:oracle:thin:@dbhost:1521:dbsid", "scott", "tiger", "oracle.jdbc.driver.OracleDriver")
Because the oracle driver will be in the GroovyClassLoader, which is down the hierarchy relative to java.lang.Class, which is up in the root classloader, the loading will fail.
The @Grab annotation seems not to support the definition of which classloader to add the dependencies to, but luckily we have programmatic access to grape too. What we need to do is something like this:
import groovy.sql.Sql
def classLoader = this.getClass().getClassLoader();
while (!classLoader.getClass().getName().equals("org.codehaus.groovy.tools.RootLoader")) {
classLoader = classLoader.getParent()
}
// force grape to use the root classloader - to ensure that Class.forName works for dependencies
groovy.grape.Grape.grab(group:'oracle', module:'ojdbc15', version:'[11.2.0.1.0,)', classLoader: classLoader)
class SqlTool {
private sql;
public SqlTool(url, driver, username, password) {
this.sql = Sql.newInstance(url, username, password, driver)
// ... do sql stuff
}
}
new SqlTool("jdbc:oracle:thin:@dbhost:1521:dbsid", "oracle.jdbc.driver.OracleDriver", "scott", "tiger")
April 19, 2010
Tags: grab, grape, groovy, jdbc Posted in: Programming 2 Comments
Easy Groovy Script Deployment with Grape
You can use groovy for lots of stuff, but many of us use it for simple, one-off administration tasks like upgrading a database schema, doing some reporting or other small tasks. Given the absolute great integration of groovy into the Java platform, a groovy script quickly ends up utilizing a bunch of jar dependencies from various open source frameworks. So, how to easy deploy a script, so that it can be started on the target platform with the least fiddling?
Use Grape!
With groovy comes Grape and the @Grab anotation. Grape is capable of downloading dependencies from maven repositories, much like maven does at compile time, but Grape can do it at execution time of the groovy script. In addition to the automatic resolving and downloading of dependencies, Grape is also capable of adding the dependencies on the script classpath before running it. Cool eh!? Here’s a small example:
package net.techper.foo.bar;
import org.jfree.chart.JFreeChart
import org.joda.time.DateMidnight
@Grapes([
@Grab(group = 'jfree', module = 'jfreechart', version = '1.0.12'),
@Grab(group = 'joda-time', module = 'joda-time', version = '1.6'),
])
class YourCoolTool {
....
}
This script use jfreechart and jodatime, both of which will be downloaded at runtime and added to classpath before the actual script execution.
Grape Internal and Config
Of course, dependencies only need to be downloaded the first time.В The downloaded files are kept in $HOME/.groovy/grapes in a maven-like repository structure.
So what about dependencies that are not in the public repository? Well, you can get them too, as long as the target environment, in which the groovy script is executed, can reach the local repository, where you have these non-public dependencies stored.
Internally, Grape is implemented with Ivy. And Grape can be configured with a Ivy configuration file. Inside the jar distribution of standard groovy lies a default Ivy config, which has some of the major maven repositories predefined. You can provide your own though, by adding a file $HOME/.groovy/grapeConfig.xml which could look something like this:
<ivysettings>
<settings defaultResolver="downloadGrapes"/>
<resolvers>
<chain name="downloadGrapes">
<filesystem name="cachedGrapes">
<ivy pattern="${user.home}/.groovy/grapes/[organisation]/[module]/ivy-[revision].xml"/>
<artifact pattern="${user.home}/.groovy/grapes/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
</filesystem>
<!-- todo add 'endorsed groovy extensions' resolver here -->
<ibiblio name="codehaus" root="http://repository.codehaus.org/" m2compatible="true"/>
<ibiblio name="ibiblio" m2compatible="true"/>
<ibiblio name="java.net2" root="http://download.java.net/maven/2/" m2compatible="true"/>
<ibiblio name="yourid" root="http://your/local/maven/repository/" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
Notice the extra ibiblio element, that is used to define a private repository!
All this makes it possible to write a small script in groovy, deploy it simply by copying it to the target environment, and then execute it at that place with no extra fuss.
April 18, 2010
Tags: deployment, grab, grape, groovy, ivy Posted in: Programming 2 Comments
Selenium RC, Safari and Blocking Browser Pop-Ups
Just a quick note here:
If you are using selenium rc to remote control a Safari on a Mac, remember to turn off pop-up blocking, or else your tests will hang in “selenium.start()” somewhere deep down in HttpURLConnection.getResponseCode() because the selenium-rc server never returns with any content.
In Safari menu, de-select “Block Pop-Up Windows”.
Let’s just say this wasted some time on me, so hope this will help someone out there in the same starter problems
February 13, 2010
Tags: mac, OSX, safari, selenium Posted in: Programming, Testing, Tools One Comment
Debugging Hibernate AbstractFlushingEventListener Errors When Batching
Here is little trick for Hibernate users.
If you have Hibernate configured to use JDBC batching, then statements in the same transaction might be batched into one or few update trips to the server. In case of any errors when performing the batch operation, you wont be able to see which statement that failed. What you will see is something in the lines of this stacktrace:
org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:94) at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66) at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167) at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:50) ... Caused by: java.sql.BatchUpdateException: ORA-00001: unique constraint (XXX) violated at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10055) at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:213) at org.apache.tomcat.dbcp.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297) at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70) at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268) ... 112 more
You can see the offending SQL exception, in this case a Oracle unique constraint violation, but you cannot see the SQL from the batch.
Hibernate uses a class called org.hibernate.util.JDBCExceptionReporter when logging JDBC exceptions. It either WARN of ERROR logs, and most people have at least ERROR level activated for something like hibernate. But, this little piece of the class is interesting:
public static void logExceptions(SQLException ex, String message) {
if ( log.isErrorEnabled() ) {
if ( log.isDebugEnabled() ) {
message = StringHelper.isNotEmpty(message) ? message : DEFAULT_EXCEPTION_MSG;
log.debug( message, ex );
}
while (ex != null) {
StringBuffer buf = new StringBuffer(30)
.append( "SQL Error: " )
.append( ex.getErrorCode() )
.append( ", SQLState: " )
.append( ex.getSQLState() );
log.warn( buf.toString() );
log.error( ex.getMessage() );
ex = ex.getNextException();
}
}
}
What is does is WARN and ERROR log a given exception and its causes. But it as has a part only executed, if DEBUG level has been turned on for this particular class/logger. It DEBUG logs the incoming parameter message and this message just so happens to be the offending SQL statement.
So, turn on DEBUG level for the org.hibernate.util.JDBCExceptionReporter logger and get more information on your failing SQL.
January 25, 2010
Posted in: Programming No Comments
Basic Auth – Just Say No
One simply should not use basic auth. Period. And I am not talking about security here. Only functionality and what you can, and most importantly cannot do with basic auth.
Okay, I know that was a bit harsh, and I do use basic auth myself sometimes. Like for instance with something dead-simple on an intranet, where we just need to shut out the masses a bit.
But what is wrong with basic auth then?
Logout Not Possible
Because the browser remember the authentication data and keep resending it for each request to the host authenticated against, there can be no logout, aside from closing the browser. Often, this is seen done with Javascript (closing the browser window), but a) it is not safe as the client can say no and b) some browsers deny it.
It Does Not Present Itself Nicely to the Client
It is the browser, that pops up a dialog. Basically, you cannot as a developer, control how this dialog looks and behaves. And it looks different in each browser. So, you cannot even determine if it needs to ask for “username”, “login” or …
Also, the application cannot show meaningful information to the client, when authentication goes wrong. This could be “you have xx tries left, before the login is locked”.
No Way To Provide Extra Functionality
Like a “change password” option, when the login has expired. Or a “remember me” option. Or whatever the application would like. Again, it is the browser, that does it all.
All this stems from the fact that it is the browser, that controls the authentication. It is the browser, that stores the authentication data and send it on with each request. And it does not have an API or the like to clear it, hook into missing auth or anything like it. The application is out of the loop.
Just say no!
December 4, 2009
Tags: http, security Posted in: Programming 7 Comments