red5/redbox rtmpt deadlock

Post your questions and bug-reports about the audio/video streaming add-on based on Red5 Media Server.

Moderators: Bax, Lapo, rjgtav

red5/redbox rtmpt deadlock

Postby projuri » 26 May 2010, 00:36

Hi,

We have switched to RTMPT protocol to traverse firewalls and after ~36 hours redbox has stopped receive requests on RTMPT port.

Those are the last messages from redbox's log file:
Code: Select all
Acceptor1 SelectChannelConnector@0.0.0.0:8080:( org.mortbay.log.invoke ) Out of threads for org.mortbay.thread.BoundedThreadPool@27e32382
DefaultQuartzScheduler_Worker-7:( org.red5.server.net.rtmp.RTMPConnection.execute ) Closing RTMPTConnection from 58.107.144.100:64373 to 114.141.198.176 (in: 4132, out: 7331624) due to too much inactivity (60006).
DefaultQuartzScheduler_Worker-5:( org.red5.server.net.rtmp.RTMPConnection.execute ) Closing RTMPTConnection from 60.229.13.140:60689 to 114.141.198.176 (in: 3610, out: 665852) due to too much inactivity (64996).


jstack has detected 8 deadlocks:
Code: Select all
Found one Java-level deadlock:
=============================
"btpool0-455 - /send/651773844/146":
  waiting to lock monitor 0x000000005422f350 (object 0x00002aaab42c8958, a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine),
  which is held by "pool-1-thread-10"
"pool-1-thread-10":
  waiting to lock monitor 0x0000000054213438 (object 0x00002aaab42c8828, a java.util.ArrayList),
  which is held by "btpool0-455 - /send/651773844/146"

Java stack information for the threads listed above:
===================================================
"btpool0-455 - /send/651773844/146":
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.stop(PlaylistSubscriberStream.java:1267)
        - waiting to lock <0x00002aaab42c8958> (a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine)
        at org.red5.server.stream.PlaylistSubscriberStream.stop(PlaylistSubscriberStream.java:298)
        at org.red5.server.stream.PlaylistSubscriberStream.removeAllItems(PlaylistSubscriberStream.java:361)
        - locked <0x00002aaab42c8828> (a java.util.ArrayList)
        at org.red5.server.stream.StreamService.play(StreamService.java:210)
        at org.red5.server.stream.StreamService.play(StreamService.java:238)
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.red5.server.service.ServiceInvoker.invoke(ServiceInvoker.java:217)
        at org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.java:187)
        at org.red5.server.net.rtmp.RTMPHandler.onInvoke(RTMPHandler.java:414)
        at org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived(BaseRTMPHandler.java:143)
        at org.red5.server.net.rtmpt.RTMPTHandler.messageReceived(RTMPTHandler.java:117)
        at org.red5.server.net.rtmpt.RTMPTServlet.handleSend(RTMPTServlet.java:387)
        at org.red5.server.net.rtmpt.RTMPTServlet.service(RTMPTServlet.java:471)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:309)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
        at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:844)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:644)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
"pool-1-thread-10":
        at org.red5.server.stream.PlaylistSubscriberStream.hasMoreItems(PlaylistSubscriberStream.java:408)
        - waiting to lock <0x00002aaab42c8828> (a java.util.ArrayList)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.stop(PlaylistSubscriberStream.java:1273)
        - locked <0x00002aaab42c8958> (a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.pullAndPush(PlaylistSubscriberStream.java:1407)
        - locked <0x00002aaab42c8958> (a org.red5.server.stream.PlaylistSubscriberStream$PlayEngine)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.access$400(PlaylistSubscriberStream.java:786)
        at org.red5.server.stream.PlaylistSubscriberStream$PlayEngine$PullAndPushRunnable.run(PlaylistSubscriberStream.java:2032)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
        at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)


SmartFoxServer 1.6.6 is being used.

Is there any fix for that problem?

Thank you
projuri
 
Posts: 7
Joined: 25 May 2010, 01:25

Postby Bax » 26 May 2010, 07:21

Actually that seems to be a problem with the Red5 server. Maybe you should Google-around to see if you can find specific support on this topic.
Also, the Red5 version which comes with SmartFoxServer is quite old. You may try to upgrade it to the latest version.
Paolo Bax
The SmartFoxServer Team
User avatar
Bax
Site Admin
 
Posts: 3846
Joined: 29 Mar 2005, 09:50
Location: Italy

Postby rock217 » 26 May 2010, 15:00

Seeing this myself a lot as well, very irritating. Google-ing yields lots of users with the same issue and zero resolutions.

PLZ HALP US!
rock217
 
Posts: 1
Joined: 26 May 2010, 14:58

Postby projuri » 30 May 2010, 22:52

Red5 upgrade has solved the issue.

Thank you, bax!
projuri
 
Posts: 7
Joined: 25 May 2010, 01:25

Postby Bax » 31 May 2010, 07:02

projuri,
could you create a new post describing the upgrade process step-by-step? This will help other users. Thank you.
Paolo Bax
The SmartFoxServer Team
User avatar
Bax
Site Admin
 
Posts: 3846
Joined: 29 Mar 2005, 09:50
Location: Italy


Return to RedBox

Who is online

Users browsing this forum: No registered users and 3 guests