Crash your Application with docx4j and WebLogic 12.1.2

Posted by nerdcoding on June 16, 2015

How to make your JavaEE 6 application unusable with the usage of docx4j and deployed on WebLogic 12.1.2.

My use case for docx4j

Docx4j is a library for creation and manipulation of Office Open XML documents. In my case I had a .docx template file and my application generated at runtime a PDF file based on this template. The business logic was inside of an EJB. Let us call it PDFGeneratorEJB. This EJB read the .docx template and inside of the template are some placeholders which are dynamically filled with data. With the combination of template and filtered placeholders a PDF file could be generated. Here is a simple JUnit test for this EJB:

public void testGeneratePDF() throws  Exception {
    byte[] pdf = new PDFGeneratorEJB().createPDF(...);

We manually create an instance of our EJB and call the PDF generation method. This method takes some parameters which are used to fill the placeholders and finally the generated PDF file is returned as an byte array.

Docx4j uses JAXB to parse the .docx template and create an in-memory object representation. The reference implementation of JAXB is part of the Metro project and is shipped with the default JDK 1.7 which I had to use. Docx4j finds this JAXB implementation on the classpath and is using it. On my local machine the PDF generation with the JUnit test executes in ca. 4 seconds. Thats not really fast, but OK.

Deployed on WebLogic 12.1.2

In the real world the PDFGeneratorEJB is part of an JavaEE 6 application running in the WebLogic application server 12.1.2. Here is where the fun begins. The WebLogic server does not use the JAXB implementation shipped with the JDK, but the EclipseLink implementation is used. The EclipseLink project also implements the JAXB specification and this project is named MOXy. WebLogic as an application server is shipped with EclipseLink and each deployed application needs to use this version of EclipseLink.

When now our deployed PDFGeneratorEJB generates a PDF file and the docx4j searches for an JAXB implementation on the classpath, the implementation of EclipseLink is found. WebLogic uses the EclipseLink version 2.4.2. In my WebLogic installation I found this in $MW_HOME/oracle_common/modules/oracle.toplink_12.1.2/eclipselink.jar.

The deployed PDFGeneratorEJB now needs ca. 2 minutes to generate a PDF file (yes I mean minutes(!) not seconds). The only difference between the deployed PDFGeneratorEJB and the JUnit test is the different JAXB implementation.

How to make the whole application unusable

But it gets worse. In my application the PDFGeneratorEJB is called (indirectly) by an message driven bean (MDB) which is triggered by JMS messages. For each incoming JMS message the PDFGeneratorEJB is called and the generated PDF is saved as BLOB in the database.

The WebLogic server creates 16 instances (default value) of that MDB inside of pool. When there are many incoming JMS messages, there are up to 16 parallel PDF generations. Because each PDF generation runs inside if the same server they are limited by the server resources. Now a PDF generation takes an average of 20 minutes for a single PDF and the server CPU is constantly at 100%. Anymore, during the PDF generation JAXB creates a huge amount of short-lived objects which leads to a lot minor garbage collections. And when now a small base of users login to the application and in the background are some PDF generations, the response times are really really slow.

Complicating is that there are many timeouts because of the long running PDF generations. Either there are transaction timeouts or, when the transaction timeout is increased, the WebLogic throws stuck threads timeouts when the execution of an MDB takes too much time. Such an timeout aborts the PDF generation, the initial JMS message is put back on the queue and over the time the amount of JMS messages is increased. Of course I can increase the transaction and stuck thread timeout. But what is a reasonable value here? 20 minutes? 30 minutes? There’s no way that’s a realistic scenario.

The root of all evil is that we are forced to use the JAXB implementation EclipseLink MOXy 2.4.2 in WebLogic 12.1.2. The next WebLogic release 12.1.3 comes with EclipseLink MOXy 2.5.2 and when we deploy our application on that WebLogic version the PDF generation is done again in a few seconds. To solve our problem there are three options:

  1. Get rid of docx4j and use another library for PDF generation which is not heavely based on JAXB.

  2. Update the WebLogic application server from 12.1.2 to 12.1.3

  3. Change the JAXB implementation inside of the WebLogic 12.1.2.

The first two solutions make the most sense, but they are not implemented quickly. To change the library of our PDF generation means a lot of coding and testing. And to update the application server also means a lot of testing. The third solution is definitely the dirtiest but also the fastest solution.

Change the JAXB implementation in WebLogic 12.1.2

I see two ways to ways to change EclipseLink MOXy 2.4.2 to EclipseLink MOXy 2.5.2 in WebLogic 12.1.2.

  1. SharedLibrary
    The WebLogic supports a concept called Shared Library. We have to create a new deployable artifact (e.g. an EAR file) which only contains the new EclipseLink version. This EAR is deployed to the WebLogic and our real application references this SharedLibrary with a <library-ref> in the deployment descriptor weblogic-application.xml

  2. APP-INF/lib
    At the EAR level of our application we create a APP-INF/lib directory which contains the new eclipselink.jar. WebLogic uses an system and an application classloader and normally classes loaded by the system classloader are preferred. This means the EclipseLink version provided by the container is used by default. To enforce to load classes from the application classloader, and therefore use the EclipseLink version form our APP-INF/lib directory, we need to set <prefer-application-packages> in the weblogic-application.xml deployment descriptor.

In my case I implemented the APP-INF/lib solution and it worked.