Skip to content

java.net.SocketException Broken Pipe Exception

For the last week, I’ve been chasing down a randomly occuring broken pipe exception in the banking app we’ve written. It was like looking for a needle in a haystack.
The Problem
Database connections were being tied up to the length of wait_timeout on the MySQL server. We’re using Hibernate for our persistence layer and C3P0 for our connection pool. During initial pilot testing, we only saw issues after a long period of time. My assumption was that the pool was not doing its job and MySQL was killing the connections. However, after ramping up the testing to a larger scale, the exception began occuring much more often.
Here is the dump I was seeing:

10:01:57,812 WARN NewPooledConnection:356 – [c3p0] A PooledConnection that has already signalled a Connection error is still in use!
10:01:57,813 WARN NewPooledConnection:357 – [c3p0] Another error has occurred [ com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: Broken pipe
STACKTRACE:
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java(Compiled Code))
at java.net.SocketOutputStream.write(SocketOutputStream.java(Compiled Code))
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java(Inlined Compiled Code))
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java(Inlined Compiled Code))
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java(Compiled Code))
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java(Inlined Compiled Code))
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java(Compiled Code))
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java(Compiled Code))
at com.mysql.jdbc.Connection.execSQL(Connection.java(Compiled Code))
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:936)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1030)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:139)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1669)
at org.hibernate.loader.Loader.doQuery(Loader.java:662)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
at org.hibernate.loader.Loader.doList(Loader.java:2150)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2029)
at org.hibernate.loader.Loader.list(Loader.java:2024)
at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:369)
at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:300)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:153)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1127)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:79)
at org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:749)
at com.bbcs.consumercard.dao.AccountDAO.getAccount(AccountDAO.java:72)
at com.bbcs.consumercard.actions.account.WebPayRequestAction.doExecute(WebPayRequestAction.java:53)
at com.bbcs.consumercard.actions.SecureBaseAction.execute(SecureBaseAction.java:38)
at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
at javax.servlet.http.HttpServlet.service(HttpServlet.java(Compiled Code))
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:199)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:282)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:744)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:674)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:866)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:570)
** END NESTED EXCEPTION **

Possible Solutions
Initially I thought the problem may lie somewhere in our code, but since this was the first time I used C3P0 for my connection pool I decided to Google the problem first. Many of the postings I found related to failures on the part of the connection pool. Based on this post, I ripped out C3P0 and implemented DBCP. That did not fix my problem, though. Some posts say it is a problem with the driver (I do not believe that it is), others blame the c3p0 connection pool (I do not).
Our Solution
In fact, our problem was sitting in our own code and was implemented by someone who was not seasoned in transactions. When you demarcate transactions, you have to explicity commit or rollback every time. Some newbies believe that queries do not have to be committed. They do. If not, the connection is left assigned in case you wanted to manipulate the resultset or do additional reads (there are other reasons, but these are the easiest to wrap your mind around).
So in our case, a transaction was begun and a query was executed. Unfortunately, the transaction was not committed in every circumstance, leaving dangling connections that later got killed by MySQL. C3P0 would later attempt to reattach to the now defunct dangling connection and kaboom – java.net.SocketException Broken PIpe would appear. Finding the spot in the code (it was in some complicated if/else logic) and correcting it solved our problem and we have not had a problem with Hibernate or C3P0 since!
Update: 7 August 2006
In the 3 days since I’ve posted this, it has already garnered a lot of page views. I didn’t realize it would be such a hit. For most of you, it should just be a matter of setting up periodic connection testing. Here are some more pointers to information if you are getting this error:
C3P0: Configuring Connection Testing
Hibernate specific properties for C3P0
More Hibernate specific configuration info
Updated 15 February 2007 – fixed link to “Hibernate Specific Properties for C3P0″

16 Comments Post a comment
  1. Nikhil #

    got to this page after all these years and it is still very helpful. thanks.

    January 9, 2013
  2. lanre #

    After 6 years, it’s still an eye opener. Thanks.

    October 2, 2012
  3. lucius #

    thank you very much, it is helpful for me.

    January 13, 2012
  4. Alfredo Mezquitic #

    I really appreciate your post…!!!.. thx

    November 28, 2008
  5. Julien #

    Thank !

    August 7, 2008
  6. nyjava #

    Thanks a lot

    June 25, 2008
  7. prashant #

    thanks for this Guidence.

    May 7, 2008
  8. Arman #

    Good post. Thanks!

    April 29, 2008
  9. Raja Nagendra Kumar #

    Short cut approach could be use
    <property name=”hibernate.connection.autocommit”>true</property>
    to make the auto commit happen.
    Regards,
    Raja Nagendra Kumar
    Java Code Audit and Refactoring Offshore company

    April 13, 2008
  10. Thank you! Your report helped me very much!!

    November 15, 2007
  11. dinakar #

    Good One!
    Thanks
    Dina.

    October 7, 2007
  12. Thank you very much! Very useful!

    September 29, 2007
  13. Yassin SIbai #

    Thank you for the info. It is very helpful.

    May 17, 2007
  14. S├ębastien Christy #

    Thanks for the info, it’s very valuable.
    The “Hibernate specific properties for C3P0″ link is broken.
    I think the correct link is : http://www.mchange.com/projects/c3p0/index.html#hibernate-specific

    February 15, 2007
  15. Ovidiu #

    cool info
    Thanks!

    December 15, 2006
  16. Naveen #

    1. java.net.SocketException:Connection reset by peer:
    JVM_recv in socket input stream read.
    2. IOException:Connection reset by peer:socket write error.
    3. IOExcepion:Broken pipe.
    All this exceptions are not thrown at the same time and might not always been thrown.
    These Exceptions are mainly caused due to the resubmission of jsp page twice. By using Input type = ‘submit’ itself submits the page not need to call from java script as form.submit. If we want to use form.submit from java script for Input type as ‘submit’ then you need to disable the submit button after one click.
    The threads running under the server may not be executing synchronously and inappropriate page submissions can cause this exceptions.

    October 11, 2006

Leave a Reply

You may use basic HTML in your comments. Your email address will not be published.

Subscribe to this comment feed via RSS