title: Scala: Akka actor not dying in Play Framework 2.2.0
body: <p>I have an example I'm building using Play Framework 2.2.0-scala that uses WebSockets to stream data to a client. The problem I'm having is that for whatever reason, one of the children of a parent Actor is not properly being shutdown. All logs indicate that it is stopping and it has shutdown, but I see it isn't actually down by publishing data to it. Here's some code, first with my controller action:</p>
  
  <pre><code>def scores(teamIds: String) = WebSocket.async[JsValue] { request =&gt;
      val teamIdsArr:Array[String] = teamIds.split(",").distinct.map { el =&gt;
          s"nfl-streaming-scores-${el}"
      }
  
      val scoresStream = Akka.system.actorOf(Props(new ScoresStream(teamIdsArr)))
      ScoresStream.join(scoresStream)
    }
  </code></pre>
  
  <p>So every time a client connects, they join <code>ScoresStream</code> which returns the respective Iteratee,Enumerator that WebSocket.async requires. The actual ScoresStream object looks like the following:</p>
  
  <pre><code>object ScoresStream {
  
    implicit val timeout = Timeout(5 seconds)
  
    def join(scoresStream:ActorRef):scala.concurrent.Future[(Iteratee[JsValue,_],Enumerator[JsValue])] = {
  
      (scoresStream ? BeginStreaming).map {
  
        case Connected(enumerator) =&gt; 
          val iteratee = Iteratee.foreach[JsValue] { _ =&gt; 
            Logger.info("Ignore iteratee input.")
          }.map { _ =&gt; 
            Logger.info("Client quitting - killing Actor.")
            scoresStream ! UnsubscribeAll
            scoresStream ! PoisonPill
          }
          (iteratee,enumerator)
  }
  </code></pre>
  
  <p>The idea here is to kill the main Actor, <code>ScoresStream</code>, when the client disconnects. I do that by using <code>scoresStream ! PoisonPill</code>. </p>
  
  <p><code>ScoresStream</code> in turn creates <code>Pub</code> and <code>Sub</code> instances which are wrappers that connect to Redis for publishing/scribing to messages, here's the Actor code:</p>
  
  <pre><code>class ScoresStream(teamIds: Array[String]) extends Actor with CreatePubSub with akka.actor.ActorLogging {
  
    val (scoresEnumerator, scoresChannel) = Concurrent.broadcast[JsValue]
  
    case class Message(kind: String, user: String, message: String)
    implicit val messageReads = Json.reads[Message]
    implicit val messageWrites = Json.writes[Message]
  
    val sub = context.child("sub") match {
      case None    =&gt; createSub(scoresChannel)
      case Some(c) =&gt; c
    }
  
    val pub = context.child("pub") match {
       case None     =&gt; createPub(teamIds)
       case Some(c)  =&gt; c
    }
  
    def receive = {
      case BeginStreaming =&gt; {
        log.info("hitting join...")
        sub ! RegisterCallback
        sub ! SubscribeChannel(teamIds)
        sender ! Connected(scoresEnumerator)
      }
  
      case UnsubscribeAll =&gt; {
        sub ! UnsubscribeChannel(teamIds)
      }
    }
  
  }
  
  trait CreatePubSub { self:Actor =&gt;
    def createSub(pChannel: Concurrent.Channel[JsValue]) = context.actorOf(Props(new Sub(pChannel)), "sub")
    def createPub(teamIds: Array[String]) = context.actorOf(Props(new Pub(teamIds)), "pub")
  }
  </code></pre>
  
  <p>Finally, here's the actual Sub Actor code: (<code>Pub</code> doesn't seem relevant here as it is shutting down fine):</p>
  
  <pre><code>class Sub(pChannel: Concurrent.Channel[JsValue]) extends Actor with CreatePublisherSubscriber with ActorLogging {
    val s = context.child("subscriber") match {
      case None    =&gt; createSubscriber
      case Some(c) =&gt; c
    }
  
    def callback(pubsub: PubSubMessage) = pubsub match {
      case E(exception) =&gt; println("Fatal error caused consumer dead. Please init new consumer reconnecting to master or connect to backup")
      case S(channel, no) =&gt; println("subscribed to " + channel + " and count = " + no)
      case U(channel, no) =&gt; println("unsubscribed from " + channel + " and count = " + no)
      case M(channel, msg) =&gt; 
        msg match {
          // exit will unsubscribe from all channels and stop subscription service
          case "exit" =&gt; 
            println("unsubscribe all ..")
            pChannel.end
            r.unsubscribe
  
          // message "+x" will subscribe to channel x
          case x if x startsWith "+" =&gt; 
            val s: Seq[Char] = x
            s match {
              case Seq('+', rest @ _*) =&gt; r.subscribe(rest.toString){ m =&gt; }
            }
  
          // message "-x" will unsubscribe from channel x
          case x if x startsWith "-" =&gt; 
            val s: Seq[Char] = x
            s match {
              case Seq('-', rest @ _*) =&gt; r.unsubscribe(rest.toString)
                                          pChannel.end
            }
  
          case x =&gt; 
           try {
              log.info("Just got a message: " + x)
              pChannel.push(Json.parse(x))
            } 
            catch {
              case ex: com.fasterxml.jackson.core.JsonParseException =&gt; {
                log.info("Malformed JSON sent.")
              }
            }
        }
    }
  
    def receive = {
      case RegisterCallback =&gt; {
        log.info("Creating a subscriber and registering callback")  
        s ! Register(callback)
      }
      case SubscribeChannel(teamIds) =&gt; {
        teamIds.foreach { x =&gt; log.info("subscribing to channel " + x + " ") }
        //sub ! Subscribe(Array("scores-5","scores-6"))
        s ! Subscribe(teamIds)
      }
      case UnsubscribeChannel(teamIds) =&gt; {
        teamIds.foreach { x =&gt; log.info("unsubscribing from channel " + x + " ") }
        s ! Unsubscribe(teamIds)
      }
      case true =&gt; println("Subscriber successfully received message.")
      case false =&gt; println("Something went wrong.")
    }
  }
  
  trait CreatePublisherSubscriber { self:Actor =&gt;
    def r = new RedisClient("localhost", 6379)
    def createSubscriber = context.actorOf(Props(new Subscriber(r)), "subscriber")
    def createPublisher = context.actorOf(Props(new Publisher(r)), "publisher")
  }
  </code></pre>
  
  <p>Now when a client connects, the startup messages look healthy:</p>
  
  <pre><code>[DEBUG] [10/20/2013 00:35:53.618] [application-akka.actor.default-dispatcher-12] [akka://application/user] now supervising Actor[akka://application/user/$c#-54456921]
  [DEBUG] [10/20/2013 00:35:53.619] [application-akka.actor.default-dispatcher-12] [akka://application/user/$c] started (com.example.stream.models.ScoresStream@131a9310)
  [DEBUG] [10/20/2013 00:35:53.620] [application-akka.actor.default-dispatcher-12] [akka://application/user/$c] now supervising Actor[akka://application/user/$c/sub#1376180991]
  [DEBUG] [10/20/2013 00:35:53.621] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c/pub/publisher] started (com.redis.Publisher@3b34c0a6)
  [DEBUG] [10/20/2013 00:35:53.622] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c/sub/subscriber] started (com.redis.Subscriber@453f0a8)
  Subscriber successfully received message.
  Subscriber successfully received message.
  [DEBUG] [10/20/2013 00:35:53.699] [application-akka.actor.default-dispatcher-19] [akka://application/user/$c/sub] started (com.example.stream.models.Sub@6165ab39)
  [DEBUG] [10/20/2013 00:35:53.699] [application-akka.actor.default-dispatcher-19] [akka://application/user/$c/sub] now supervising Actor[akka://application/user/$c/sub/subscriber#-1562348862]
  subscribed to nfl-streaming-scores-5 and count = 1
  [DEBUG] [10/20/2013 00:35:53.699] [application-akka.actor.default-dispatcher-12] [akka://application/user/$c] now supervising Actor[akka://application/user/$c/pub#-707418539]
  [INFO] [10/20/2013 00:35:53.700] [application-akka.actor.default-dispatcher-12] [akka://application/user/$c] hitting join...
  [INFO] [10/20/2013 00:35:53.700] [application-akka.actor.default-dispatcher-23] [akka://application/user/$c/sub] Creating a subscriber and registering callback
  [INFO] [10/20/2013 00:35:53.700] [application-akka.actor.default-dispatcher-23] [akka://application/user/$c/sub] subscribing to channel nfl-streaming-scores-5 
  [DEBUG] [10/20/2013 00:35:53.700] [application-akka.actor.default-dispatcher-18] [akka://application/user/$c/pub] started (com.example.stream.models.Pub@48007a17)
  [DEBUG] [10/20/2013 00:35:53.703] [application-akka.actor.default-dispatcher-18] [akka://application/user/$c/pub] now supervising Actor[akka://application/user/$c/pub/publisher#1509054514]
  </code></pre>
  
  <p>And disconnecting looks healthy:</p>
  
  <pre><code>[info] application - Client quitting - killing Actor.
  unsubscribed from nfl-streaming-scores-5 and count = 0
  [DEBUG] [10/20/2013 00:37:51.696] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c] received AutoReceiveMessage Envelope(PoisonPill,Actor[akka://application/deadLetters])
  [INFO] [10/20/2013 00:37:51.696] [application-akka.actor.default-dispatcher-25] [akka://application/user/$c/sub] unsubscribing from channel nfl-streaming-scores-5 
  [DEBUG] [10/20/2013 00:37:51.696] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c] stopping
  [DEBUG] [10/20/2013 00:37:51.697] [application-akka.actor.default-dispatcher-25] [akka://application/user/$c/sub] stopping
  [DEBUG] [10/20/2013 00:37:51.697] [application-akka.actor.default-dispatcher-25] [akka://application/user/$c/pub/publisher] stopped
  [DEBUG] [10/20/2013 00:37:51.697] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c/sub/subscriber] stopped
  [DEBUG] [10/20/2013 00:37:51.697] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c/sub] stopped
  [INFO] [10/20/2013 00:37:51.697] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c/sub] Message [java.lang.Boolean] from Actor[akka://application/user/$c/sub/subscriber#-1562348862] to Actor[akka://application/user/$c/sub#1376180991] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
  [DEBUG] [10/20/2013 00:37:51.699] [application-akka.actor.default-dispatcher-26] [akka://application/user/$c/pub] stopping
  [DEBUG] [10/20/2013 00:37:51.699] [application-akka.actor.default-dispatcher-26] [akka://application/user/$c/pub] stopped
  [DEBUG] [10/20/2013 00:37:51.699] [application-akka.actor.default-dispatcher-17] [akka://application/user/$c] stopped
  </code></pre>
  
  <p>And here's the problem, after the client has disconnected, I'm going to send a message that the currently shutdown Actor was subscribed to:</p>
  
  <pre><code>redis-cli publish "nfl-streaming-scores-5" "{\"test\":\"message\"}"
  </code></pre>
  
  <p>and here it is showing up, when it shouldn't be, this Actor should technically be dead. Other Actors that were around before receive the message also, ones labeled with $a/$b. I can confirm no other clients are connected. </p>
  
  <pre><code>[INFO] [10/20/2013 00:38:33.097] [Thread-7] [akka://application/user/$c/sub] Just got a message: {"test":"message"}
  </code></pre>
  
  <p>What's also an odd indicator is that address names never get re-used. I keep seeing a trend like the following of names spawn when I disconnect/connect:</p>
  
  <pre><code>akka://application/user/$c
  akka://application/user/$d
  akka://application/user/$e
  </code></pre>
  
  <p>Never see old references get re-used.</p>
  
  <p>My assumption here is that the connection to Redis is not being cleanly closed. It doesn't explain why logs say the Actor has stopped yet still exist, but I definitely see connections established to redis after running <code>netstat</code> even after all Actors are presumably dead. When I stop the application completely from running, those connections clear up. It's as if unsubscribe is silently failing and that's keeping the Actor alive and also the connection, which is really bad for multiple reasons because eventually the system will run out of file descriptors and/or have memory leaks. Is there something obvious here that I'm doing wrong?</p>