问:
Why is there a handshake failure when trying to run TLS over TLS with this code?
I tried to implement a protocol that can run TLS over TLS using twisted.protocols.tls, an interface to OpenSSL using a memory BIO.

I implemented this as a protocol wrapper that mostly looks like a regular TCP transport, but which has startTLS and stopTLS methods for adding and removing a layer of TLS respectively.  This works fine for the first layer of TLS.  It also works fine if I run it over a "native" Twisted TLS transport.  However, if I try to add a second TLS layer using the startTLS method provided by this wrapper, there's immediately a handshake error and the connection ends up in some unknown unusable state.

The wrapper and the two helpers that let it work looks like this:

```<code>from twisted.python.components import proxyForInterface
from twisted.internet.error import ConnectionDone
from twisted.internet.interfaces import ITCPTransport, IProtocol
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory

class TransportWithoutDisconnection(proxyForInterface(ITCPTransport)):
    """
    A proxy for a normal transport that disables actually closing the connection.
    This is necessary so that when TLSMemoryBIOProtocol notices the SSL EOF it
    doesn't actually close the underlying connection.

    All methods except loseConnection are proxied directly to the real transport.
    """
    def loseConnection(self):
        pass

class ProtocolWithoutConnectionLost(proxyForInterface(IProtocol)):
    """
    A proxy for a normal protocol which captures clean connection shutdown
    notification and sends it to the TLS stacking code instead of the protocol.
    When TLS is shutdown cleanly, this notification will arrive.  Instead of telling
    the protocol that the entire connection is gone, the notification is used to
    unstack the TLS code in OnionProtocol and hidden from the wrapped protocol.  Any
    other kind of connection shutdown (SSL handshake error, network hiccups, etc) are
    treated as real problems and propagated to the wrapped protocol.
    """
    def connectionLost(self, reason):
        if reason.check(ConnectionDone):
            self.onion._stopped()
        else:
            super(ProtocolWithoutConnectionLost, self).connectionLost(reason)

class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol, it can run over
    any other ITransport.  As a transport, it implements stackable TLS.  That is,
    whatever application traffic is generated by the protocol running on top of
    OnionProtocol can be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS conversation can
    be encapsulated in yet *another* TLS conversation.

    Each layer of TLS can use different connection parameters, such as keys, ciphers,
    certificate requirements, etc.  At the remote end of this connection, each has to
    be decrypted separately, starting at the outermost and working in.  OnionProtocol
    can do this itself, of course, just as it can encrypt each layer starting with the
    innermost.
    """
    def makeConnection(self, transport):
        self._tlsStack = []
        ProtocolWrapper.makeConnection(self, transport)

    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given contextFactory.

        If *client* is True, this side of the connection will be an SSL client.
        Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of the SSL handshake
        were received by the protocol running on top of OnionProtocol, they must be
        passed here as the **bytes** parameter.
        """
        # First, create a wrapper around the application-level protocol
        # (wrappedProtocol) which can catch connectionLost and tell this OnionProtocol
        # about it.  This is necessary to pop from _tlsStack when the outermost TLS
        # layer stops.
        connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
        connLost.onion = self
        # Construct a new TLS layer, delivering events and application data to the
        # wrapper just created.
        tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        # Push the previous transport and protocol onto the stack so they can be
        # retrieved when this new TLS layer stops.
        self._tlsStack.append((self.transport, self.wrappedProtocol))

        # Create a transport for the new TLS layer to talk to.  This is a passthrough
        # to the OnionProtocol's current transport, except for capturing loseConnection
        # to avoid really closing the underlying connection.
        transport = TransportWithoutDisconnection(self.transport)

        # Make the new TLS layer the current protocol and transport.
        self.wrappedProtocol = self.transport = tlsProtocol

        # And connect the new TLS layer to the previous outermost transport.
        self.transport.makeConnection(transport)

        # If the application accidentally got some bytes from the TLS handshake, deliver
        # them to the new TLS layer.
        if bytes is not None:
            self.wrappedProtocol.dataReceived(bytes)

    def stopTLS(self):
        """
        Remove a layer of TLS.
        """
        # Just tell the current TLS layer to shut down.  When it has done so, we'll get
        # notification in *_stopped*.
        self.transport.loseConnection()

    def _stopped(self):
        # A TLS layer has completely shut down.  Throw it away and move back to the
        # TLS layer it was wrapping (or possibly back to the original non-TLS
        # transport).
        self.transport, self.wrappedProtocol = self._tlsStack.pop()
</code>```

I have simple client and server programs for exercising this, available from launchpad (bzr branch lp:~exarkun/+junk/onion).  When I use it to call the startTLS method above twice, with no intervening call to stopTLS,  this OpenSSL error comes up:

```<code>OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]
</code>```

Why do things go wrong?

答:
There are at least two problems with OnionProtocol:

The innermost TLSMemoryBIOProtocol becomes the wrappedProtocol, when it should be the outermost;

ProtocolWithoutConnectionLost does not pop any TLSMemoryBIOProtocols off OnionProtocol's stack, because connectionLost is only called after a FileDescriptors doRead or doWrite methods return a reason for disconnection.

We can't solve the first problem without changing the way OnionProtocol manages its stack, and we can't solve the second until we figure out the new stack implementation.  Unsurprisingly, the correct design is a direct consequence of how data flows within Twisted, so we'll start with some data flow analysis.

Twisted represents an established connection with an instance of either twisted.internet.tcp.Server or twisted.internet.tcp.Client. Since the only interactivity in our program happens in stoptls_client, we'll only consider the data flow to and from a Client instance.

Let's warm up with a minimal LineReceiver client that echoes back lines received from a local server on port 9999:

```<code>from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, task

class LineReceiver(basic.LineReceiver):
    def lineReceived(self, line):
        self.sendLine(line)

def main(reactor):
    clientEndpoint = endpoints.clientFromString(
        reactor, "tcp:localhost:9999")
    connected = clientEndpoint.connect(
        protocol.ClientFactory.forProtocol(LineReceiver))
    def waitForever(_):
        return defer.Deferred()
    return connected.addCallback(waitForever)

task.react(main)
</code>```

Once the established connection's established, a Client becomes our LineReceiver protocol's transport and mediates input and output:

New data from the server causes the reactor to call the Client's doRead method, which in turn passes what it's received to LineReceiver's dataReceived method. Finally, LineReceiver.dataReceived calls LineReceiver.lineReceived when at least one line is available.

Our application sends a line of data back to the server by calling LineReceiver.sendLine.  This calls write on the transport bound to the protocol instance, which is the same Client instance that handled incoming data.  Client.write arranges for the data to be sent by the reactor, while Client.doWrite actually sends the data over the socket.

We're ready to look at the behaviors of an OnionClient that never calls startTLS:

OnionClients are wrapped in OnionProtocols, which are the crux of our attempt at nested TLS.  As a subclass of twisted.internet.policies.ProtocolWrapper, an instance of OnionProtocol is a kind of protocol-transport sandwich; it presents itself as a protocol to a lower-level transport and as a transport to a protocol it wraps through a masquerade established at connection time by a WrappingFactory.

Now, Client.doRead calls OnionProtocol.dataReceived, which proxies the data through to OnionClient.  As OnionClient's transport, OnionProtocol.write accepts lines to send from OnionClient.sendLine and proxies them down to Client, its own transport.  This is the normal interaction between a ProtocolWrapper, its wrapped protocol, and its own transport, so naturally data flows to and from each without any trouble.

OnionProtocol.startTLS does something different.  It attempts to interpose a new ProtocolWrapper — which happens to be a TLSMemoryBIOProtocol — between an established protocol-transport pair.  This seems easy enough: a ProtocolWrapper stores the upper-level protocol as its wrappedProtocol attribute, and proxies write and other attributes down to its own transport.  startTLS should be able to inject a new TLSMemoryBIOProtocol that wraps OnionClient into the connection by patching that instance over its own wrappedProtocol and transport:

```<code>def startTLS(self):
    ...
    connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
    connLost.onion = self
    # Construct a new TLS layer, delivering events and application data to the
    # wrapper just created.
    tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)

    # Push the previous transport and protocol onto the stack so they can be
    # retrieved when this new TLS layer stops.
    self._tlsStack.append((self.transport, self.wrappedProtocol))
    ...
    # Make the new TLS layer the current protocol and transport.
    self.wrappedProtocol = self.transport = tlsProtocol
</code>```

Here's the flow of data after the first call to startTLS:

As expected, new data delivered to OnionProtocol.dataReceived is routed to the TLSMemoryBIOProtocol stored on the _tlsStack, which passes the decrypted plaintext to OnionClient.dataReceived.  OnionClient.sendLine also passes its data to TLSMemoryBIOProtocol.write, which encrypts it and sends the resulting ciphertext to OnionProtocol.write and then Client.write.

Unfortunately this scheme fails after a second call to startTLS.  The root cause is this line:

```<code>    self.wrappedProtocol = self.transport = tlsProtocol
</code>```

Each call to startTLS replaces the wrappedProtocol with the innermost TLSMemoryBIOProtocol, even though the data received by Client.doRead was encrypted by the outermost:

The transports, however, are nested correctly.  OnionClient.sendLine can only call its transport's write — that is, OnionProtocol.write — so OnionProtocol should replace its transport with the innermost TLSMemoryBIOProtocol to ensure writes are successively nested inside additional layers of encryption.

The solution, then, is to ensure that data flows through the first TLSMemoryBIOProtocol on the _tlsStack to the next one in turn, so that each layer of encryption is peeled off in the reverse order it was applied:

Representing _tlsStack as a list seems less natural given this new requirement.  Fortunately, representing the incoming data flow linearly suggests a new data structure:

Both the buggy and correct flow of incoming data resemble a singly-linked list, with wrappedProtocol serving as ProtocolWrappers next links and protocol serving as Client's.  The list should grow downward from OnionProtocol and always end with OnionClient.  The bug occurs because that ordering invariant is violated.

A singly-linked list is fine for pushing protocols onto the stack but awkward for popping them off, because it requires traversal downwards from its head to the node to remove.  Of course, this traversal happens every time data's received, so the concern is the complexity implied by an additional traversal rather than worst-case time complexity.  Fortunately, the list is actually doubly linked:

The transport attribute links each nested protocol with its predecessor, so that transport.write can layer on successively lower levels of encryption before finally sending the data across the network.  We have two sentinels to aid in managing the list: Client must always be at the top and OnionClient must always be at the bottom.

Putting the two together, we end up with this:

```<code>from twisted.python.components import proxyForInterface
from twisted.internet.interfaces import ITCPTransport
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory

class PopOnDisconnectTransport(proxyForInterface(ITCPTransport)):
    """
    L{TLSMemoryBIOProtocol.loseConnection} shuts down the TLS session
    and calls its own transport's C{loseConnection}.  A zero-length
    read also calls the transport's C{loseConnection}.  This proxy
    uses that behavior to invoke a C{pop} callback when a session has
    ended.  The callback is invoked exactly once because
    C{loseConnection} must be idempotent.
    """
    def __init__(self, pop, **kwargs):
        super(PopOnDisconnectTransport, self).__init__(**kwargs)
        self._pop = pop

    def loseConnection(self):
        self._pop()
        self._pop = lambda: None

class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol,
    it can run over any other ITransport.  As a transport, it
    implements stackable TLS.  That is, whatever application traffic
    is generated by the protocol running on top of OnionProtocol can
    be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS
    conversation can be encapsulated in yet *another* TLS
    conversation.

    Each layer of TLS can use different connection parameters, such as
    keys, ciphers, certificate requirements, etc.  At the remote end
    of this connection, each has to be decrypted separately, starting
    at the outermost and working in.  OnionProtocol can do this
    itself, of course, just as it can encrypt each layer starting with
    the innermost.
    """

    def __init__(self, *args, **kwargs):
        ProtocolWrapper.__init__(self, *args, **kwargs)
        # The application level protocol is the sentinel at the tail
        # of the linked list stack of protocol wrappers.  The stack
        # begins at this sentinel.
        self._tailProtocol = self._currentProtocol = self.wrappedProtocol

    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given
        contextFactory.

        If *client* is True, this side of the connection will be an
        SSL client.  Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of
        the SSL handshake were received by the protocol running on top
        of OnionProtocol, they must be passed here as the **bytes**
        parameter.
        """
        # The newest TLS session is spliced in between the previous
        # and the application protocol at the tail end of the list.
        tlsProtocol = TLSMemoryBIOProtocol(None, self._tailProtocol, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        if self._currentProtocol is self._tailProtocol:
            # This is the first and thus outermost TLS session.  The
            # transport is the immutable sentinel that no startTLS or
            # stopTLS call will move within the linked list stack.
            # The wrappedProtocol will remain this outermost session
            # until it's terminated.
            self.wrappedProtocol = tlsProtocol
            nextTransport = PopOnDisconnectTransport(
                original=self.transport,
                pop=self._pop
            )
            # Store the proxied transport as the list's head sentinel
            # to enable an easy identity check in _pop.
            self._headTransport = nextTransport
        else:
            # This a later TLS session within the stack.  The previous
            # TLS session becomes its transport.
            nextTransport = PopOnDisconnectTransport(
                original=self._currentProtocol,
                pop=self._pop
            )

        # Splice the new TLS session into the linked list stack.
        # wrappedProtocol serves as the link, so the protocol at the
        # current position takes our new TLS session as its
        # wrappedProtocol.
        self._currentProtocol.wrappedProtocol = tlsProtocol
        # Move down one position in the linked list.
        self._currentProtocol = tlsProtocol
        # Expose the new, innermost TLS session as the transport to
        # the application protocol.
        self.transport = self._currentProtocol
        # Connect the new TLS session to the previous transport.  The
        # transport attribute also serves as the previous link.
        tlsProtocol.makeConnection(nextTransport)

        # Left over bytes are part of the latest handshake.  Pass them
        # on to the innermost TLS session.
        if bytes is not None:
            tlsProtocol.dataReceived(bytes)

    def stopTLS(self):
        self.transport.loseConnection()

    def _pop(self):
        pop = self._currentProtocol
        previous = pop.transport
        # If the previous link is the head sentinel, we've run out of
        # linked list.  Ensure that the application protocol, stored
        # as the tail sentinel, becomes the wrappedProtocol, and the
        # head sentinel, which is the underlying transport, becomes
        # the transport.
        if previous is self._headTransport:
            self._currentProtocol = self.wrappedProtocol = self._tailProtocol
            self.transport = previous
        else:
            # Splice out a protocol from the linked list stack.  The
            # previous transport is a PopOnDisconnectTransport proxy,
            # so first retrieve proxied object off its original
            # attribute.
            previousProtocol = previous.original
            # The previous protocol's next link becomes the popped
            # protocol's next link
            previousProtocol.wrappedProtocol = pop.wrappedProtocol
            # Move up one position in the linked list.
            self._currentProtocol = previousProtocol
            # Expose the new, innermost TLS session as the transport
            # to the application protocol.
            self.transport = self._currentProtocol

class OnionFactory(WrappingFactory):
    """
    A L{WrappingFactory} that overrides
    L{WrappingFactory.registerProtocol} and
    L{WrappingFactory.unregisterProtocol}.  These methods store in and
    remove from a dictionary L{ProtocolWrapper} instances.  The
    C{transport} patching done as part of the linked-list management
    above causes the instances' hash to change, because the
    C{__hash__} is proxied through to the wrapped transport.  They're
    not essential to this program, so the easiest solution is to make
    them do nothing.
    """
    protocol = OnionProtocol

    def registerProtocol(self, protocol):
        pass

    def unregisterProtocol(self, protocol):
        pass
</code>```

(This is also available on GitHub.)

The solution to the second problem lies in PopOnDisconnectTransport.  The original code attempted to pop off a TLS session from the stack via connectionLost, but because only a closed file descriptor causes connectionLost to be called, it failed to remove stopped TLS sessions that didn't close the underlying socket.

At the time of this writing, TLSMemoryBIOProtocol calls its transport's loseConnection in exactly two places: _shutdownTLS and _tlsShutdownFinished.  _shutdownTLS is called on active closes (loseConnection, abortConnection, unregisterProducer and after loseConnection and all pending writes have been flushed), while _tlsShutdownFinished is called on passive closes (handshake failures, empty reads, read errors, and write errors).  This all means that both sides of a closed connection can pop stopped TLS sessions off the stack during loseConnection. PopOnDisconnectTransport does this idempotently because loseConnection is generally idempotent, and TLSMemoryBIOProtocol certainly expects it to be.

The downside to putting stack management logic in loseConnection is that it depends on the particulars of TLSMemoryBIOProtocol's implementation.  A generalized solution would require new APIs across many levels of Twisted.

Until then, we're stuck with another example of Hyrum's Law.

Body:
I tried to implement a protocol that can run TLS over TLS using twisted.protocols.tls, an interface to OpenSSL using a memory BIO.

I implemented this as a protocol wrapper that mostly looks like a regular TCP transport, but which has startTLS and stopTLS methods for adding and removing a layer of TLS respectively.  This works fine for the first layer of TLS.  It also works fine if I run it over a "native" Twisted TLS transport.  However, if I try to add a second TLS layer using the startTLS method provided by this wrapper, there's immediately a handshake error and the connection ends up in some unknown unusable state.

The wrapper and the two helpers that let it work looks like this:

from twisted.python.components import proxyForInterface
from twisted.internet.error import ConnectionDone
from twisted.internet.interfaces import ITCPTransport, IProtocol
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory

class TransportWithoutDisconnection(proxyForInterface(ITCPTransport)):
    """
    A proxy for a normal transport that disables actually closing the connection.
    This is necessary so that when TLSMemoryBIOProtocol notices the SSL EOF it
    doesn't actually close the underlying connection.

    All methods except loseConnection are proxied directly to the real transport.
    """
    def loseConnection(self):
        pass

class ProtocolWithoutConnectionLost(proxyForInterface(IProtocol)):
    """
    A proxy for a normal protocol which captures clean connection shutdown
    notification and sends it to the TLS stacking code instead of the protocol.
    When TLS is shutdown cleanly, this notification will arrive.  Instead of telling
    the protocol that the entire connection is gone, the notification is used to
    unstack the TLS code in OnionProtocol and hidden from the wrapped protocol.  Any
    other kind of connection shutdown (SSL handshake error, network hiccups, etc) are
    treated as real problems and propagated to the wrapped protocol.
    """
    def connectionLost(self, reason):
        if reason.check(ConnectionDone):
            self.onion._stopped()
        else:
            super(ProtocolWithoutConnectionLost, self).connectionLost(reason)

class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol, it can run over
    any other ITransport.  As a transport, it implements stackable TLS.  That is,
    whatever application traffic is generated by the protocol running on top of
    OnionProtocol can be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS conversation can
    be encapsulated in yet *another* TLS conversation.

    Each layer of TLS can use different connection parameters, such as keys, ciphers,
    certificate requirements, etc.  At the remote end of this connection, each has to
    be decrypted separately, starting at the outermost and working in.  OnionProtocol
    can do this itself, of course, just as it can encrypt each layer starting with the
    innermost.
    """
    def makeConnection(self, transport):
        self._tlsStack = []
        ProtocolWrapper.makeConnection(self, transport)

    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given contextFactory.

        If *client* is True, this side of the connection will be an SSL client.
        Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of the SSL handshake
        were received by the protocol running on top of OnionProtocol, they must be
        passed here as the **bytes** parameter.
        """
        # First, create a wrapper around the application-level protocol
        # (wrappedProtocol) which can catch connectionLost and tell this OnionProtocol
        # about it.  This is necessary to pop from _tlsStack when the outermost TLS
        # layer stops.
        connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
        connLost.onion = self
        # Construct a new TLS layer, delivering events and application data to the
        # wrapper just created.
        tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        # Push the previous transport and protocol onto the stack so they can be
        # retrieved when this new TLS layer stops.
        self._tlsStack.append((self.transport, self.wrappedProtocol))

        # Create a transport for the new TLS layer to talk to.  This is a passthrough
        # to the OnionProtocol's current transport, except for capturing loseConnection
        # to avoid really closing the underlying connection.
        transport = TransportWithoutDisconnection(self.transport)

        # Make the new TLS layer the current protocol and transport.
        self.wrappedProtocol = self.transport = tlsProtocol

        # And connect the new TLS layer to the previous outermost transport.
        self.transport.makeConnection(transport)

        # If the application accidentally got some bytes from the TLS handshake, deliver
        # them to the new TLS layer.
        if bytes is not None:
            self.wrappedProtocol.dataReceived(bytes)

    def stopTLS(self):
        """
        Remove a layer of TLS.
        """
        # Just tell the current TLS layer to shut down.  When it has done so, we'll get
        # notification in *_stopped*.
        self.transport.loseConnection()

    def _stopped(self):
        # A TLS layer has completely shut down.  Throw it away and move back to the
        # TLS layer it was wrapping (or possibly back to the original non-TLS
        # transport).
        self.transport, self.wrappedProtocol = self._tlsStack.pop()

I have simple client and server programs for exercising this, available from launchpad (bzr branch lp:~exarkun/+junk/onion).  When I use it to call the startTLS method above twice, with no intervening call to stopTLS,  this OpenSSL error comes up:

OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]

Why do things go wrong?

Body:
There are at least two problems with OnionProtocol:

The innermost TLSMemoryBIOProtocol becomes the wrappedProtocol, when it should be the outermost;

ProtocolWithoutConnectionLost does not pop any TLSMemoryBIOProtocols off OnionProtocol's stack, because connectionLost is only called after a FileDescriptors doRead or doWrite methods return a reason for disconnection.

We can't solve the first problem without changing the way OnionProtocol manages its stack, and we can't solve the second until we figure out the new stack implementation.  Unsurprisingly, the correct design is a direct consequence of how data flows within Twisted, so we'll start with some data flow analysis.

Twisted represents an established connection with an instance of either twisted.internet.tcp.Server or twisted.internet.tcp.Client. Since the only interactivity in our program happens in stoptls_client, we'll only consider the data flow to and from a Client instance.

Let's warm up with a minimal LineReceiver client that echoes back lines received from a local server on port 9999:

from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, task

class LineReceiver(basic.LineReceiver):
    def lineReceived(self, line):
        self.sendLine(line)

def main(reactor):
    clientEndpoint = endpoints.clientFromString(
        reactor, "tcp:localhost:9999")
    connected = clientEndpoint.connect(
        protocol.ClientFactory.forProtocol(LineReceiver))
    def waitForever(_):
        return defer.Deferred()
    return connected.addCallback(waitForever)

task.react(main)

Once the established connection's established, a Client becomes our LineReceiver protocol's transport and mediates input and output:

New data from the server causes the reactor to call the Client's doRead method, which in turn passes what it's received to LineReceiver's dataReceived method. Finally, LineReceiver.dataReceived calls LineReceiver.lineReceived when at least one line is available.

Our application sends a line of data back to the server by calling LineReceiver.sendLine.  This calls write on the transport bound to the protocol instance, which is the same Client instance that handled incoming data.  Client.write arranges for the data to be sent by the reactor, while Client.doWrite actually sends the data over the socket.

We're ready to look at the behaviors of an OnionClient that never calls startTLS:

OnionClients are wrapped in OnionProtocols, which are the crux of our attempt at nested TLS.  As a subclass of twisted.internet.policies.ProtocolWrapper, an instance of OnionProtocol is a kind of protocol-transport sandwich; it presents itself as a protocol to a lower-level transport and as a transport to a protocol it wraps through a masquerade established at connection time by a WrappingFactory.

Now, Client.doRead calls OnionProtocol.dataReceived, which proxies the data through to OnionClient.  As OnionClient's transport, OnionProtocol.write accepts lines to send from OnionClient.sendLine and proxies them down to Client, its own transport.  This is the normal interaction between a ProtocolWrapper, its wrapped protocol, and its own transport, so naturally data flows to and from each without any trouble.

OnionProtocol.startTLS does something different.  It attempts to interpose a new ProtocolWrapper — which happens to be a TLSMemoryBIOProtocol — between an established protocol-transport pair.  This seems easy enough: a ProtocolWrapper stores the upper-level protocol as its wrappedProtocol attribute, and proxies write and other attributes down to its own transport.  startTLS should be able to inject a new TLSMemoryBIOProtocol that wraps OnionClient into the connection by patching that instance over its own wrappedProtocol and transport:

def startTLS(self):
    ...
    connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
    connLost.onion = self
    # Construct a new TLS layer, delivering events and application data to the
    # wrapper just created.
    tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)

    # Push the previous transport and protocol onto the stack so they can be
    # retrieved when this new TLS layer stops.
    self._tlsStack.append((self.transport, self.wrappedProtocol))
    ...
    # Make the new TLS layer the current protocol and transport.
    self.wrappedProtocol = self.transport = tlsProtocol

Here's the flow of data after the first call to startTLS:

As expected, new data delivered to OnionProtocol.dataReceived is routed to the TLSMemoryBIOProtocol stored on the _tlsStack, which passes the decrypted plaintext to OnionClient.dataReceived.  OnionClient.sendLine also passes its data to TLSMemoryBIOProtocol.write, which encrypts it and sends the resulting ciphertext to OnionProtocol.write and then Client.write.

Unfortunately this scheme fails after a second call to startTLS.  The root cause is this line:

    self.wrappedProtocol = self.transport = tlsProtocol

Each call to startTLS replaces the wrappedProtocol with the innermost TLSMemoryBIOProtocol, even though the data received by Client.doRead was encrypted by the outermost:

The transports, however, are nested correctly.  OnionClient.sendLine can only call its transport's write — that is, OnionProtocol.write — so OnionProtocol should replace its transport with the innermost TLSMemoryBIOProtocol to ensure writes are successively nested inside additional layers of encryption.

The solution, then, is to ensure that data flows through the first TLSMemoryBIOProtocol on the _tlsStack to the next one in turn, so that each layer of encryption is peeled off in the reverse order it was applied:

Representing _tlsStack as a list seems less natural given this new requirement.  Fortunately, representing the incoming data flow linearly suggests a new data structure:

Both the buggy and correct flow of incoming data resemble a singly-linked list, with wrappedProtocol serving as ProtocolWrappers next links and protocol serving as Client's.  The list should grow downward from OnionProtocol and always end with OnionClient.  The bug occurs because that ordering invariant is violated.

A singly-linked list is fine for pushing protocols onto the stack but awkward for popping them off, because it requires traversal downwards from its head to the node to remove.  Of course, this traversal happens every time data's received, so the concern is the complexity implied by an additional traversal rather than worst-case time complexity.  Fortunately, the list is actually doubly linked:

The transport attribute links each nested protocol with its predecessor, so that transport.write can layer on successively lower levels of encryption before finally sending the data across the network.  We have two sentinels to aid in managing the list: Client must always be at the top and OnionClient must always be at the bottom.

Putting the two together, we end up with this:

from twisted.python.components import proxyForInterface
from twisted.internet.interfaces import ITCPTransport
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory

class PopOnDisconnectTransport(proxyForInterface(ITCPTransport)):
    """
    L{TLSMemoryBIOProtocol.loseConnection} shuts down the TLS session
    and calls its own transport's C{loseConnection}.  A zero-length
    read also calls the transport's C{loseConnection}.  This proxy
    uses that behavior to invoke a C{pop} callback when a session has
    ended.  The callback is invoked exactly once because
    C{loseConnection} must be idempotent.
    """
    def __init__(self, pop, **kwargs):
        super(PopOnDisconnectTransport, self).__init__(**kwargs)
        self._pop = pop

    def loseConnection(self):
        self._pop()
        self._pop = lambda: None

class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol,
    it can run over any other ITransport.  As a transport, it
    implements stackable TLS.  That is, whatever application traffic
    is generated by the protocol running on top of OnionProtocol can
    be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS
    conversation can be encapsulated in yet *another* TLS
    conversation.

    Each layer of TLS can use different connection parameters, such as
    keys, ciphers, certificate requirements, etc.  At the remote end
    of this connection, each has to be decrypted separately, starting
    at the outermost and working in.  OnionProtocol can do this
    itself, of course, just as it can encrypt each layer starting with
    the innermost.
    """

    def __init__(self, *args, **kwargs):
        ProtocolWrapper.__init__(self, *args, **kwargs)
        # The application level protocol is the sentinel at the tail
        # of the linked list stack of protocol wrappers.  The stack
        # begins at this sentinel.
        self._tailProtocol = self._currentProtocol = self.wrappedProtocol

    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given
        contextFactory.

        If *client* is True, this side of the connection will be an
        SSL client.  Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of
        the SSL handshake were received by the protocol running on top
        of OnionProtocol, they must be passed here as the **bytes**
        parameter.
        """
        # The newest TLS session is spliced in between the previous
        # and the application protocol at the tail end of the list.
        tlsProtocol = TLSMemoryBIOProtocol(None, self._tailProtocol, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        if self._currentProtocol is self._tailProtocol:
            # This is the first and thus outermost TLS session.  The
            # transport is the immutable sentinel that no startTLS or
            # stopTLS call will move within the linked list stack.
            # The wrappedProtocol will remain this outermost session
            # until it's terminated.
            self.wrappedProtocol = tlsProtocol
            nextTransport = PopOnDisconnectTransport(
                original=self.transport,
                pop=self._pop
            )
            # Store the proxied transport as the list's head sentinel
            # to enable an easy identity check in _pop.
            self._headTransport = nextTransport
        else:
            # This a later TLS session within the stack.  The previous
            # TLS session becomes its transport.
            nextTransport = PopOnDisconnectTransport(
                original=self._currentProtocol,
                pop=self._pop
            )

        # Splice the new TLS session into the linked list stack.
        # wrappedProtocol serves as the link, so the protocol at the
        # current position takes our new TLS session as its
        # wrappedProtocol.
        self._currentProtocol.wrappedProtocol = tlsProtocol
        # Move down one position in the linked list.
        self._currentProtocol = tlsProtocol
        # Expose the new, innermost TLS session as the transport to
        # the application protocol.
        self.transport = self._currentProtocol
        # Connect the new TLS session to the previous transport.  The
        # transport attribute also serves as the previous link.
        tlsProtocol.makeConnection(nextTransport)

        # Left over bytes are part of the latest handshake.  Pass them
        # on to the innermost TLS session.
        if bytes is not None:
            tlsProtocol.dataReceived(bytes)

    def stopTLS(self):
        self.transport.loseConnection()

    def _pop(self):
        pop = self._currentProtocol
        previous = pop.transport
        # If the previous link is the head sentinel, we've run out of
        # linked list.  Ensure that the application protocol, stored
        # as the tail sentinel, becomes the wrappedProtocol, and the
        # head sentinel, which is the underlying transport, becomes
        # the transport.
        if previous is self._headTransport:
            self._currentProtocol = self.wrappedProtocol = self._tailProtocol
            self.transport = previous
        else:
            # Splice out a protocol from the linked list stack.  The
            # previous transport is a PopOnDisconnectTransport proxy,
            # so first retrieve proxied object off its original
            # attribute.
            previousProtocol = previous.original
            # The previous protocol's next link becomes the popped
            # protocol's next link
            previousProtocol.wrappedProtocol = pop.wrappedProtocol
            # Move up one position in the linked list.
            self._currentProtocol = previousProtocol
            # Expose the new, innermost TLS session as the transport
            # to the application protocol.
            self.transport = self._currentProtocol

class OnionFactory(WrappingFactory):
    """
    A L{WrappingFactory} that overrides
    L{WrappingFactory.registerProtocol} and
    L{WrappingFactory.unregisterProtocol}.  These methods store in and
    remove from a dictionary L{ProtocolWrapper} instances.  The
    C{transport} patching done as part of the linked-list management
    above causes the instances' hash to change, because the
    C{__hash__} is proxied through to the wrapped transport.  They're
    not essential to this program, so the easiest solution is to make
    them do nothing.
    """
    protocol = OnionProtocol

    def registerProtocol(self, protocol):
        pass

    def unregisterProtocol(self, protocol):
        pass

(This is also available on GitHub.)

The solution to the second problem lies in PopOnDisconnectTransport.  The original code attempted to pop off a TLS session from the stack via connectionLost, but because only a closed file descriptor causes connectionLost to be called, it failed to remove stopped TLS sessions that didn't close the underlying socket.

At the time of this writing, TLSMemoryBIOProtocol calls its transport's loseConnection in exactly two places: _shutdownTLS and _tlsShutdownFinished.  _shutdownTLS is called on active closes (loseConnection, abortConnection, unregisterProducer and after loseConnection and all pending writes have been flushed), while _tlsShutdownFinished is called on passive closes (handshake failures, empty reads, read errors, and write errors).  This all means that both sides of a closed connection can pop stopped TLS sessions off the stack during loseConnection. PopOnDisconnectTransport does this idempotently because loseConnection is generally idempotent, and TLSMemoryBIOProtocol certainly expects it to be.

The downside to putting stack management logic in loseConnection is that it depends on the particulars of TLSMemoryBIOProtocol's implementation.  A generalized solution would require new APIs across many levels of Twisted.

Until then, we're stuck with another example of Hyrum's Law.


来源: stackexchange

问:
Angular directive dependencies
Might be a very basic question, but if I am using a particular version of a dependency, for example 1.x of dependency1, in my angular directive, and I package this directive to be used as a third party directive in an app that uses version 2.x of the same dependency1, how will the behavior be? I want to understand how shared directives work with multiple apps that might use a different version of a dependency that the directive is using.

EDIT
I should probably make myself clear that I’m not asking about versions of angular. Let’s say I’m using bootstrap v3.3.7 in my directive whereas another app that uses my directive is using v3.1.1 of bootstrap. I want to understand how this would work and if there would be any conflicts.

答:
If you want / need to have a hybrid solution between versions I would recommend you have a look at the upgrade section of the documentation.  They explain what you need / can do to have an hybrid solution running angularjs and Angular.

Angular Docs Upgrade

ContentLicense: CC BY-SA 3.0

Title: Angular directive dependencies


问:
DDD - How does the domain receive client data
I'm planning to refactor a legacy .NET project that I'm maintaining, there are mainly 4 projects involved:

My.Company.Web - The presentation layer. ASP.NET MVC.

My.Company.Service - The service layer, defines service interfaces and contains command handlers. Invoked by the web layer.

My.Company.DTO - The assembly that contains DTO classes for data transfer between the domain and web layers.

My.Company.Domain - The domain layer.

My.Company.Data - The persistence layer.

When user submits a form to update a person's details, the process looks somewhat like this:

Using the submitted data, controller constructs a PersonDetailsDTO which consists of multiple simple fields to hold FirstName, LastName, Age and etc, and a complex type object called AddressDTO that holds address details: var dto = new PersonDetails(...);

Controller calls the service layer with the DTO:

```<code>_personService.UpdateDetails(dto);
</code>```

In the service:

```<code>public void UpdatePerson(PersonDetailsDTO dto)
{
    var person = _personRepository.GetById(dto.PersonId);
    person.Update(dto);
    _personRepository.Update(person);
}
</code>```

In the Person domain:

```<code>public class Person : AggregateRoot
{
    private string FirstName;
    .... // other fields
    private Address Address;

    public void Update(PersonDetailsDTO dto)
    {
        FirstName = dto.FirstName;
        .... // other fields
        Address.Update(dto.AddressDTO); // update the value object - address
    }
}
</code>```

As you can see here, the Person domain references the PersonDetailsDTO. This feels wrong to me as DTOs are means to present the data in the domain to its clients while not exposing the domain itself. and the domain should not be concerned with how it's presented to the clients.

Also IMO the DTOs should live in the My.Company.Service project as it's the service that defines the operation/data transfer contracts.

But then how would the service pass the details on to the domain? Would it have to be a loooong list of parameters: person.Update(firstName, lastName, age, gender, street1, street2, state, postcode...). This certainly looks wrong to me.

Or does the domain have to define yet another set of it's own "DTOs" to receive client data?

I have gone through a LOT of tutorials and SO posts and no one seem to share my concern, so I'm starting to think I'm missing something here, could someone point me to the right direction?

答:
You need to implement Object to Object mapping. It means that Service layer is responsible for creating and populating domain objects based on DTO by mapping values from the DTO instance received from client.

You can take advantage of Automapper library to simplify object-to-object mapping.
Of course it can be served for domain to DTO if needed.

Body:
I'm planning to refactor a legacy .NET project that I'm maintaining, there are mainly 4 projects involved:

My.Company.Web - The presentation layer. ASP.NET MVC.

My.Company.Service - The service layer, defines service interfaces and contains command handlers. Invoked by the web layer.

My.Company.DTO - The assembly that contains DTO classes for data transfer between the domain and web layers.

My.Company.Domain - The domain layer.

My.Company.Data - The persistence layer.

When user submits a form to update a person's details, the process looks somewhat like this:

Using the submitted data, controller constructs a PersonDetailsDTO which consists of multiple simple fields to hold FirstName, LastName, Age and etc, and a complex type object called AddressDTO that holds address details: var dto = new PersonDetails(...);

Controller calls the service layer with the DTO:

_personService.UpdateDetails(dto);

In the service:

public void UpdatePerson(PersonDetailsDTO dto)
{
    var person = _personRepository.GetById(dto.PersonId);
    person.Update(dto);
    _personRepository.Update(person);
}

In the Person domain:

public class Person : AggregateRoot
{
    private string FirstName;
    .... // other fields
    private Address Address;

    public void Update(PersonDetailsDTO dto)
    {
        FirstName = dto.FirstName;
        .... // other fields
        Address.Update(dto.AddressDTO); // update the value object - address
    }
}

As you can see here, the Person domain references the PersonDetailsDTO. This feels wrong to me as DTOs are means to present the data in the domain to its clients while not exposing the domain itself. and the domain should not be concerned with how it's presented to the clients.

Also IMO the DTOs should live in the My.Company.Service project as it's the service that defines the operation/data transfer contracts.

But then how would the service pass the details on to the domain? Would it have to be a loooong list of parameters: person.Update(firstName, lastName, age, gender, street1, street2, state, postcode...). This certainly looks wrong to me.

Or does the domain have to define yet another set of it's own "DTOs" to receive client data?

I have gone through a LOT of tutorials and SO posts and no one seem to share my concern, so I'm starting to think I'm missing something here, could someone point me to the right direction?


问:
How to zoom out from Google maps in Android Emulator?
I am testing a map based app on the emulator. I have zoomed into the map by double clicking with the mouse on the screen, but can not zoom out.

Please tell me how to zoom out?

答:
Above answer didn't work.

For me on Windows these worked -

Trackpad - Press and hold Ctrl + double click and hold on a second
click + drag in and out the cursor to zoom out and in respectively.

Trackpad/Mouse - Press and hold Ctrl + click left button + drag in
and out the cursor to zoom out and in respectively.

Touch screen - Press and hold Ctrl + drag in and out on the screen
to zoom out and in respectively.

ContentLicense: CC BY-SA 4.0


问:
Get updated IndexPath for visible item after UICollectionView has scrolled
I have a collectionView with different periods (day to year) for user to choose. Each item's size is equal to the size of collectionView itself.

```<code>@IBOutlet weak var periodCollectionView: UICollectionView!
@IBOutlet weak var itemLabel: UILabel!

let collectionValues = ["Day", "Week", "Month", "Year"]
var currentVisibleItem: Int?

override func viewDidLoad() {
    super.viewDidLoad()

}

func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
    return 4
}

func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {

    guard let cell = collectionView.dequeueReusableCell(withReuseIdentifier: "PeriodName", for: indexPath) as? DisplayPeriodCollectionViewCell
    else {
        fatalError("Unable to cast collection view cell as DisplayPeriodCollectionViewCell")
    }

    cell.periodName.text = collectionValues[indexPath.item]

    return cell

}

func collectionView(_ collectionView: UICollectionView, layout collectionViewLayout: UICollectionViewLayout, sizeForItemAt indexPath: IndexPath) -> CGSize {
    let height = collectionView.frame.height
    let width = collectionView.frame.width

    return CGSize(width: width, height: height)
}
</code>```

What i am trying to do is get an info label to display the index of current visible item after collectionView has scrolled. Her is the code:

```<code>func scrollViewDidEndDecelerating(_ scrollView: UIScrollView) {

    let centerPoint = CGPoint(x: periodCollectionView.center.x + periodCollectionView.contentOffset.x, y: 1.0)
    print("centerPoint: \(centerPoint)")

    if let currentItemIndexPath = periodCollectionView.indexPathForItem(at: centerPoint) {
        currentVisibleItem = currentItemIndexPath.item
        print("index of current item: \(currentVisibleItem!)")
        itemLabel.text = "\(currentVisibleItem!)"
    } else {
        print("could not get index of current item...")
    }

}
</code>```

I've added print statements to check if values are defined correctly at runtime, and they show everything is fine. The problem is that label does not always update its text after scrolling ends, but instead waits for new scrolling to occur:

I can't figure out why this is happening - setting a breakpoint at scrollViewDidEndDecelerating shows that all code gets executed, so what's wrong with the label text?

UPD. Not-updating label text appears to be a Simulator issue - when running on device, label updates correctly.

答:
Try this </p>

```<code>override func scrollViewDidEndDecelerating(_ scrollView: UIScrollView) {
    let currentIndex = scrollView.contentOffset.x / CGFloat((itemWidth + interitemSpacing / 2))
    print(currentIndex)
}
</code>```

Body:
I have a collectionView with different periods (day to year) for user to choose. Each item's size is equal to the size of collectionView itself.

@IBOutlet weak var periodCollectionView: UICollectionView!
@IBOutlet weak var itemLabel: UILabel!

let collectionValues = ["Day", "Week", "Month", "Year"]
var currentVisibleItem: Int?

override func viewDidLoad() {
    super.viewDidLoad()

}

func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
    return 4
}

func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {

    guard let cell = collectionView.dequeueReusableCell(withReuseIdentifier: "PeriodName", for: indexPath) as? DisplayPeriodCollectionViewCell
    else {
        fatalError("Unable to cast collection view cell as DisplayPeriodCollectionViewCell")
    }

    cell.periodName.text = collectionValues[indexPath.item]

    return cell

}

func collectionView(_ collectionView: UICollectionView, layout collectionViewLayout: UICollectionViewLayout, sizeForItemAt indexPath: IndexPath) -> CGSize {
    let height = collectionView.frame.height
    let width = collectionView.frame.width

    return CGSize(width: width, height: height)
}

What i am trying to do is get an info label to display the index of current visible item after collectionView has scrolled. Her is the code:

func scrollViewDidEndDecelerating(_ scrollView: UIScrollView) {

    let centerPoint = CGPoint(x: periodCollectionView.center.x + periodCollectionView.contentOffset.x, y: 1.0)
    print("centerPoint: \(centerPoint)")

    if let currentItemIndexPath = periodCollectionView.indexPathForItem(at: centerPoint) {
        currentVisibleItem = currentItemIndexPath.item
        print("index of current item: \(currentVisibleItem!)")
        itemLabel.text = "\(currentVisibleItem!)"
    } else {
        print("could not get index of current item...")
    }

}

I've added print statements to check if values are defined correctly at runtime, and they show everything is fine. The problem is that label does not always update its text after scrolling ends, but instead waits for new scrolling to occur:

I can't figure out why this is happening - setting a breakpoint at scrollViewDidEndDecelerating shows that all code gets executed, so what's wrong with the label text?

UPD. Not-updating label text appears to be a Simulator issue - when running on device, label updates correctly.

Body:
Try this

override func scrollViewDidEndDecelerating(_ scrollView: UIScrollView) {
    let currentIndex = scrollView.contentOffset.x / CGFloat((itemWidth + interitemSpacing / 2))
    print(currentIndex)
}


问:
How to only add header if value is present?
My Lambda function returns {data: {...}} body to API Gateway but it can also return {data: {...}, cookie: ...} if user is logging in.

I added Header Mapping to API Gateway that adds that cookie value to header:

```<code>Set-Cookie: integration.response.body.cookie
</code>```

Now I end up with empty set-cookie headers if response body has no cookie. This point also applies to every other header added via Header Mapping.

Is there a way to set header only when value in response body is present?

答:
Lambda Proxy Integrations will afford you the dynamic control of response headers you require.

Your Lambda function will need to return the entire HTTP response, including status code, body and headers.

http://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-output-format

Currently there is no way to dynamically map headers other than to use Proxy Integrations.

Body:
My Lambda function returns {data: {...}} body to API Gateway but it can also return {data: {...}, cookie: ...} if user is logging in.

I added Header Mapping to API Gateway that adds that cookie value to header:

Set-Cookie: integration.response.body.cookie

Now I end up with empty set-cookie headers if response body has no cookie. This point also applies to every other header added via Header Mapping.

Is there a way to set header only when value in response body is present?


问:
Is std::ifstream significantly slower than FILE?
I've been informed that my library is slower than it should be, on the order of 30+ times too slow parsing a particular file (text file, size 326 kb). The user suggested that it may be that I'm using std::ifstream (presumably instead of FILE).

I'd rather not blindly rewrite, so I thought I'd check here first, since my guess would be the bottleneck is elsewhere. I'm reading character by character, so the only functions I'm using are get(), peek(), and tellg()/seekg().

Update:

I profiled, and got confusing output - gprof didn't appear to think that it took so long. I rewrote the program to read the entire file into a buffer first, and it sped up by about 100x. I think the problem may have been the tellg()/seekg() that took a long time, but gprof may have been unable to see that for some reason. In any case, ifstream does not appear to buffer the entire file, even for this size.

答:
All benchmarks are evil. Just profile your code for the data you expect.

I performed an I/O performance comparison between Ruby, Python, Perl, C++ once. For my data, languages versions, etc C++'s variant was several times slower (it was a big suprise at that time).

OwnerDisplayName: Jesse Beder

ContentLicense: CC BY-SA 2.5

LastEditorDisplayName: Jesse Beder

OwnerDisplayName: J.F. Sebastian


问:
Why does my app throw an `android.permission.REBOOT SecurityException`?
I wrote an Android app that uses android.os.PowerManager.reboot(), and I added <uses-permission android:name="android.permission.REBOOT" /> in my AndroidManifest.xml. However, when I run the app, it always throws the following exception:

```<code>java.lang.SecurityException: Neither user 10039 nor current process has android.permission.REBOOT. at android.os.Parcel.readException(Parcel.java:1247)
at android.os.Parcel.readException(Parcel.java:1247)
at android.os.Parcel.readException(Parcel.java:1235)
at android.os.IPowerManager$Stub$Proxy.reboot(IPowerManager.java:427)
at android.os.PowerManager.reboot(PowerManager.java:481)
at Test.testPower(Test.java:374)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
at android.os.Parcel.readException(Parcel.java:1247)
at android.os.Parcel.readException(Parcel.java:1235)
at android.os.IPowerManager$Stub$Proxy.reboot(IPowerManager.java:427)
at android.os.PowerManager.reboot(PowerManager.java:481)
at com.fsl.cts.FSLPlaybackTest.testPower(FSLPlaybackTest.java:374)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
</code>```

Is the anything I'm doing wrong, or does froyo have some problem with rebooting?

答:
It can't be done in stock android phones. But, as I posted in other similar question, if the phone is rooted, then you have no problem at all:

```<code>try {
    Runtime.getRuntime().exec("su");
    Runtime.getRuntime().exec("reboot"); }
catch (IOException e) { }
</code>```

For an explanation, follow the link above.

Body:
I wrote an Android app that uses android.os.PowerManager.reboot(), and I added <uses-permission android:name="android.permission.REBOOT" /> in my AndroidManifest.xml. However, when I run the app, it always throws the following exception:

java.lang.SecurityException: Neither user 10039 nor current process has android.permission.REBOOT. at android.os.Parcel.readException(Parcel.java:1247)
at android.os.Parcel.readException(Parcel.java:1247)
at android.os.Parcel.readException(Parcel.java:1235)
at android.os.IPowerManager$Stub$Proxy.reboot(IPowerManager.java:427)
at android.os.PowerManager.reboot(PowerManager.java:481)
at Test.testPower(Test.java:374)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)
at android.os.Parcel.readException(Parcel.java:1247)
at android.os.Parcel.readException(Parcel.java:1235)
at android.os.IPowerManager$Stub$Proxy.reboot(IPowerManager.java:427)
at android.os.PowerManager.reboot(PowerManager.java:481)
at com.fsl.cts.FSLPlaybackTest.testPower(FSLPlaybackTest.java:374)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:520)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)

Is the anything I'm doing wrong, or does froyo have some problem with rebooting?

Body:
It can't be done in stock android phones. But, as I posted in other similar question, if the phone is rooted, then you have no problem at all:

try {
    Runtime.getRuntime().exec("su");
    Runtime.getRuntime().exec("reboot"); }
catch (IOException e) { }

For an explanation, follow the link above.


问:
Unpacking values from list in Python 2.7
Python Newbie here:
Given a list and a value, how can I unpack the list into separate objects and prepend a value.  Length of the array isn't known, unlike in code below:

```<code>x = [1,2,3,4]
y = 0,x
y
</code>```

Current Output:

```<code>(0, [1, 2, 3, 4])
</code>```

Desired Output:

```<code>(0, 1, 2, 3, 4)
</code>```

I know it can be done easily in Python 3 using y = 0,*x but how can it be done in Python 2.7?

Thanks

答:
Insert y into x and make x Tuple

```<code>x = [1,2,3,4]
y = 0
x.insert(0,y)
print(tuple(x))
</code>```

output

```<code>(0, 1, 2, 3, 4)
</code>```

Body:
Python Newbie here:
Given a list and a value, how can I unpack the list into separate objects and prepend a value.  Length of the array isn't known, unlike in code below:

x = [1,2,3,4]
y = 0,x
y

Current Output:

(0, [1, 2, 3, 4])

Desired Output:

(0, 1, 2, 3, 4)

I know it can be done easily in Python 3 using y = 0,*x but how can it be done in Python 2.7?

Thanks

Body:
Insert y into x and make x Tuple

x = [1,2,3,4]
y = 0
x.insert(0,y)
print(tuple(x))

output

(0, 1, 2, 3, 4)


问:
How do I remove the last comma from a string using PHP?
I am using a loop to get values from my database and my result is like:

```<code>'name', 'name2', 'name3',
</code>```

And I want it like this:

```<code>'name', 'name2', 'name3'
</code>```

I want to remove the comma after the last value of the loop.

答:
its as simple as:</p>

```<code>$commaseparated_string = name,name2,name3,;
$result = rtrim($commaseparated_string,',');
</code>```

Body:
I am using a loop to get values from my database and my result is like:

'name', 'name2', 'name3',

And I want it like this:

'name', 'name2', 'name3'

I want to remove the comma after the last value of the loop.

Body:
its as simple as:

$commaseparated_string = name,name2,name3,;
$result = rtrim($commaseparated_string,',');


问:
Mobile Detect error too many redirects
I am using a plugin which detects if the device is mobile or any pc

so here is the CODE

```<code>require_once ('Mobile-Detect-2.8.27/Mobile_Detect.php');

$detect = new Mobile_Detect;
// Any mobile device (phones or tablets).

if ( $detect->isMobile() ) {
}else{
header("Location:../login");
}
// Any tablet device.
if( $detect->isTablet() ){
}
else{
header("Location:../login");
}

// Exclude tablets.
if( $detect->isMobile() && !$detect->isTablet() ){
}else{
header("Location:../login");
}

// Check for a specific platform with the help of the magic methods:
if( $detect->isiOS() ){
}else{
header("Location:../login");
}

if( $detect->isAndroidOS() ){
}else{
header("Location:../login");
}
</code>```

This redirects as you can see to another page.

But when i use this my phone,Chrome shows a error

THIS PAGE ISN'T WORKING

Redirect too many times
What can be wrong ?

答:
Be sure to no output before any header and use exit; after each header

```<code>  header("Location:../login");
  exit;
</code>```

Title: Mobile Detect error too many redirects

Body:
I am using a plugin which detects if the device is mobile or any pc

so here is the CODE

require_once ('Mobile-Detect-2.8.27/Mobile_Detect.php');

$detect = new Mobile_Detect;
// Any mobile device (phones or tablets).

if ( $detect->isMobile() ) {
}else{
header("Location:../login");
}
// Any tablet device.
if( $detect->isTablet() ){
}
else{
header("Location:../login");
}

// Exclude tablets.
if( $detect->isMobile() && !$detect->isTablet() ){
}else{
header("Location:../login");
}

// Check for a specific platform with the help of the magic methods:
if( $detect->isiOS() ){
}else{
header("Location:../login");
}

if( $detect->isAndroidOS() ){
}else{
header("Location:../login");
}

This redirects as you can see to another page.

But when i use this my phone,Chrome shows a error

THIS PAGE ISN'T WORKING

Redirect too many times
What can be wrong ?

Body:
Be sure to no output before any header and use exit; after each header

  header("Location:../login");
  exit;


问:
Destroy and Spawn(Respawn) Objects in Unity 3D
I have a game project. You have a ball that you can control with arrow keys. Purpose is, you will collecting an object. Object will destroy when i going over it, and respawn in random location of ground. Btw, you need to do that in 30 seconds. I couldnt destroy and respawn part. May you help me? (C#)</p>

答: You can use collisions to destroy object. Run over it and yield. After that you can create couple of spawn points (for more randomness you can move the spawn points).


问:
Rails: Relation does not exist for reference with class name in production
I created two references in a migration that are aliases for a reference to my User table:

```<code>class CreateInvitations < ActiveRecord::Migration[5.0]
  def change
    create_table :invitations do |t|
      t.references :owner, references: :user, foreign_key: true # the owner
      t.references :invitee, references: :user, foreign_key: true # the invitee
      t.references :core_bot, foreign_key: true # the associated page (core_bot_active)
      t.string :email
      t.string :token

      t.timestamps

    end
  end
end
</code>```

In my User model:

```<code>  has_many :invitations, foreign_key: :owner_id
  has_many :invitations, foreign_key: :invitee_id, dependent: :destroy
</code>```

In my Invitation model:

```<code>  belongs_to :owner, class_name: :User
  belongs_to :invitee, class_name: :User
</code>```

Everything works well in development but when I try to migrate in production with Heroku heroku run rake db:migrate, I get the following error:

PG::UndefinedTable: ERROR:  relation "owners" does not exist : CREATE
  TABLE "invitations" ("id" serial primary key, "owner_id" integer,
  "invitee_id" integer, "core_bot_id" integer, "email" character
  varying, "token" character varying, "created_at" timestamp NOT NULL,
  "updated_at" timestamp NOT NULL, CONSTRAINT "fk_rails_59e24979a9"
  FOREIGN KEY ("owner_id")   REFERENCES "owners" ("id") , CONSTRAINT
  "fk_rails_00204dc74b" FOREIGN KEY ("invitee_id")   REFERENCES
  "invitees" ("id") , CONSTRAINT "fk_rails_34505bdb65" FOREIGN KEY
  ("core_bot_id")   REFERENCES "core_bots" ("id") )

I tried without references: :user but I get the same error.

Any idea what's wrong here?

答:
Your development is probably an sqLite database but Heroku uses PostgreSQL and the interpretation of the migration is generating a foregn key to owners

Write the migration like this instead...

```<code>class CreateInvitations < ActiveRecord::Migration[5.0]
  def change
    create_table :invitations do |t|
      t.references :owner, index: true # the owner
      t.references :invitee, index: true # the invitee
      t.references :core_bot, foreign_key: true # the associated page (core_bot_active)
      t.string :email
      t.string :token

      t.timestamps

    end
    add_foreign_key :invitations, :users, column: :owner_id
    add_foreign_key :invitations, :users, column: :invitee_id
  end
end
</code>```

It's one of the risks with developing using a different database product than the production implementation. Migrations may not work exactly the same. If planning to deploy to Heroku you should look at using postgreSQL in development.

Body:
I created two references in a migration that are aliases for a reference to my User table:

class CreateInvitations < ActiveRecord::Migration[5.0]
  def change
    create_table :invitations do |t|
      t.references :owner, references: :user, foreign_key: true # the owner
      t.references :invitee, references: :user, foreign_key: true # the invitee
      t.references :core_bot, foreign_key: true # the associated page (core_bot_active)
      t.string :email
      t.string :token

      t.timestamps

    end
  end
end

In my User model:

  has_many :invitations, foreign_key: :owner_id
  has_many :invitations, foreign_key: :invitee_id, dependent: :destroy

In my Invitation model:

  belongs_to :owner, class_name: :User
  belongs_to :invitee, class_name: :User

Everything works well in development but when I try to migrate in production with Heroku heroku run rake db:migrate, I get the following error:

PG::UndefinedTable: ERROR:  relation "owners" does not exist : CREATE
  TABLE "invitations" ("id" serial primary key, "owner_id" integer,
  "invitee_id" integer, "core_bot_id" integer, "email" character
  varying, "token" character varying, "created_at" timestamp NOT NULL,
  "updated_at" timestamp NOT NULL, CONSTRAINT "fk_rails_59e24979a9"
  FOREIGN KEY ("owner_id")   REFERENCES "owners" ("id") , CONSTRAINT
  "fk_rails_00204dc74b" FOREIGN KEY ("invitee_id")   REFERENCES
  "invitees" ("id") , CONSTRAINT "fk_rails_34505bdb65" FOREIGN KEY
  ("core_bot_id")   REFERENCES "core_bots" ("id") )

I tried without references: :user but I get the same error.

Any idea what's wrong here?

Body:
Your development is probably an sqLite database but Heroku uses PostgreSQL and the interpretation of the migration is generating a foregn key to owners

Write the migration like this instead...

class CreateInvitations < ActiveRecord::Migration[5.0]
  def change
    create_table :invitations do |t|
      t.references :owner, index: true # the owner
      t.references :invitee, index: true # the invitee
      t.references :core_bot, foreign_key: true # the associated page (core_bot_active)
      t.string :email
      t.string :token

      t.timestamps

    end
    add_foreign_key :invitations, :users, column: :owner_id
    add_foreign_key :invitations, :users, column: :invitee_id
  end
end

It's one of the risks with developing using a different database product than the production implementation. Migrations may not work exactly the same. If planning to deploy to Heroku you should look at using postgreSQL in development.


问:
Explain this snippet which finds the maximum of two integers without using if-else or any other comparison operator?
Find the maximum of two numbers. You should not use if-else or any other comparison operator. I found this question on online bulletin board, so i thought i should ask in StackOverflow

EXAMPLE
Input: 5, 10
Output: 10

I found this solution, can someone help me understand these lines of code

```<code>int getMax(int a, int b) {
    int c = a - b;
    int k = (c >> 31) & 0x1;
    int max = a - k * c;
    return max;
}
</code>```

答:
```<code>int getMax(int a, int b) {
    int c = a - b;
    int k = (c >> 31) & 0x1;
    int max = a - k * c;
    return max;
}
</code>```

Let's dissect this.  This first line appears to be straightforward - it stores the difference of a and b. This value is negative if a < b and is nonnegative otherwise. But there's actually a bug here - if the difference of the numbers a and b is so big that it can't fit into an integer, this will lead to undefined behavior - oops! So let's assume that doesn't happen here.

In the next line, which is

```<code>int k = (c >> 31) & 0x1;
</code>```

the idea is to check if the value of c is negative.  In virtually all modern computers, numbers are stored in a format called two's complement in which the highest bit of the number is 0 if the number is positive and 1 if the number is negative.  Moreover, most ints are 32 bits.  (c >> 31) shifts the number down 31 bits, leaving the highest bit of the number in the spot for the lowest bit.  The next step of taking this number and ANDing it with 1 (whose binary representation is 0 everywhere except the last bit) erases all the higher bits and just gives you the lowest bit.  Since the lowest bit of c >> 31 is the highest bit of c, this reads the highest bit of c as either 0 or 1.  Since the highest bit is 1 iff c is 1, this is a way of checking whether c is negative (1) or positive (0).  Combining this reasoning with the above, k is 1 if a < b and is 0 otherwise.

The final step is to do this:

```<code>int max = a - k * c;
</code>```

If a < b, then k == 1 and k * c = c = a - b, and so

```<code>a - k * c = a - (a - b) = a - a + b = b
</code>```

Which is the correct max, since a < b.  Otherwise, if a >= b, then k == 0 and

```<code>a - k * c = a - 0 = a
</code>```

Which is also the correct max.

Body:
Find the maximum of two numbers. You should not use if-else or any other comparison operator. I found this question on online bulletin board, so i thought i should ask in StackOverflow

EXAMPLE
Input: 5, 10
Output: 10

I found this solution, can someone help me understand these lines of code

int getMax(int a, int b) {
    int c = a - b;
    int k = (c >> 31) & 0x1;
    int max = a - k * c;
    return max;
}

Body:
int getMax(int a, int b) {
    int c = a - b;
    int k = (c >> 31) & 0x1;
    int max = a - k * c;
    return max;
}

Let's dissect this.  This first line appears to be straightforward - it stores the difference of a and b. This value is negative if a < b and is nonnegative otherwise. But there's actually a bug here - if the difference of the numbers a and b is so big that it can't fit into an integer, this will lead to undefined behavior - oops! So let's assume that doesn't happen here.

In the next line, which is

int k = (c >> 31) & 0x1;

the idea is to check if the value of c is negative.  In virtually all modern computers, numbers are stored in a format called two's complement in which the highest bit of the number is 0 if the number is positive and 1 if the number is negative.  Moreover, most ints are 32 bits.  (c >> 31) shifts the number down 31 bits, leaving the highest bit of the number in the spot for the lowest bit.  The next step of taking this number and ANDing it with 1 (whose binary representation is 0 everywhere except the last bit) erases all the higher bits and just gives you the lowest bit.  Since the lowest bit of c >> 31 is the highest bit of c, this reads the highest bit of c as either 0 or 1.  Since the highest bit is 1 iff c is 1, this is a way of checking whether c is negative (1) or positive (0).  Combining this reasoning with the above, k is 1 if a < b and is 0 otherwise.

The final step is to do this:

int max = a - k * c;

If a < b, then k == 1 and k * c = c = a - b, and so

a - k * c = a - (a - b) = a - a + b = b

Which is the correct max, since a < b.  Otherwise, if a >= b, then k == 0 and

a - k * c = a - 0 = a

Which is also the correct max.


问:
Keep all lines of a list with identical beginning (Notepad++)
From a list, how to keep all occurrences of those lines only whose "first part or beginning" (defined from the beginning of the line to the ^ character) is present in other lines? (The pattern of lines in the list: beginning-of-line^rest_of_line_012345)

The type of characters, length, etc. after the ^ is irrelevant (but needs to be kept). Every line has only one (1) ^ character. The "beginning" string that determines identity must be present in the same (analogous) position in other lines (i.e., from the beginning of the line to ^, and must be exact match). (Lines contain characters that trouble regex, such as \/()*., so these need to be summarily escaped.)

For example: Original list:

```<code>abc^123
0xyz^xxx
aaa-123^123
aaa-12^0xyz
0xyz^098
00xyz^098
0xyz^x111xx
</code>```

Keep all occurrences of lines with identical first part:

```<code>0xyz^xxx
0xyz^098
0xyz^x111xx
</code>```

This elegant script by @Lars Fischer    ((.*)\R(\2\R?)+)*\K.* (after pre-sorting) keeps all occurrences of duplicate lines, but it considers the entire line (it was designed to do so).

In this Q, I am looking for a solution that considers only the "beginning" of the line to see if it occurs more than once, and if yes, then keep the entire line. Any guidance?

答:
You said in comments that a perl script is OK for you.

```<code>#!/usr/bin/perl
use Modern::Perl;

my %values;
my $file = 'path/to/file';
open my $fh, '<', $file or die "unable to open '$file': $!";
while(<$fh>) {
    chomp;
    # get the prefix value
    my ($prefix) = split('\^', $_);
    # push in array the whole line in hash with the prefix as key
    push @{$values{$prefix}}, $_;
}
foreach (keys %values) {
    # skip the prefix tat have only one line
    next if scalar @{$values{$_}} == 1;
    local $" = "\n";
    say "@{$values{$_}}";
}
</code>```

Output:

```<code>0xyz^xxx
0xyz^098
0xyz^x111xx
</code>```

Body:
From a list, how to keep all occurrences of those lines only whose "first part or beginning" (defined from the beginning of the line to the ^ character) is present in other lines? (The pattern of lines in the list: beginning-of-line^rest_of_line_012345)

The type of characters, length, etc. after the ^ is irrelevant (but needs to be kept). Every line has only one (1) ^ character. The "beginning" string that determines identity must be present in the same (analogous) position in other lines (i.e., from the beginning of the line to ^, and must be exact match). (Lines contain characters that trouble regex, such as \/()*., so these need to be summarily escaped.)

For example: Original list:

abc^123
0xyz^xxx
aaa-123^123
aaa-12^0xyz
0xyz^098
00xyz^098
0xyz^x111xx

Keep all occurrences of lines with identical first part:

0xyz^xxx
0xyz^098
0xyz^x111xx

This elegant script by @Lars Fischer    ((.*)\R(\2\R?)+)*\K.* (after pre-sorting) keeps all occurrences of duplicate lines, but it considers the entire line (it was designed to do so).

In this Q, I am looking for a solution that considers only the "beginning" of the line to see if it occurs more than once, and if yes, then keep the entire line. Any guidance?

Body:
You said in comments that a perl script is OK for you.

#!/usr/bin/perl
use Modern::Perl;

my %values;
my $file = 'path/to/file';
open my $fh, '<', $file or die "unable to open '$file': $!";
while(<$fh>) {
    chomp;
    # get the prefix value
    my ($prefix) = split('\^', $_);
    # push in array the whole line in hash with the prefix as key
    push @{$values{$prefix}}, $_;
}
foreach (keys %values) {
    # skip the prefix tat have only one line
    next if scalar @{$values{$_}} == 1;
    local $" = "\n";
    say "@{$values{$_}}";
}

Output:

0xyz^xxx
0xyz^098
0xyz^x111xx


问:
Docker build not setting env variables correctly?
I've got this Dockerfile:

```<code>FROM ubuntu:latest

RUN apt-get update && apt-get install -y \
        curl \
        python3-dev \
        python3-setuptools \
        python3-pip

RUN rm -rf /var/lib/apt/lists/* && \
        apt-get clean

ADD . /usr/src/index-server
WORKDIR /usr/src/index-server

RUN pip3 install --upgrade pip
RUN pip3 install -r requirements.txt

EXPOSE 8000

ENV LC_CTYPE=C.UTF-8
ENV FLASK_APP=app_server.py

CMD /usr/local/bin/flask run --host=0.0.0.0
</code>```

It builds fine. But when I try to run it, I run into a Python error, which I tracked to incorrectly set locale values.

On my host machine locale | grep -E 'LC_ALL|LC_CTYPE|LANG' returns:

```<code>LANG="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_ALL=
</code>```

But when I try the same with the docker instance (docker run -t 657c402ec253 locale | grep -E 'LC_ALL|LC_CTYPE|LANG'), I get this:

```<code>LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_ALL=
</code>```

If I use docker's -e flag to set the env variable, it works fine:

```<code>$ docker run -e LC_CTYPE=C.UTF-8 -t 657c402ec253 locale | grep -E 'LC_ALL|LC_CTYPE|LANG'

LANG=
LANGUAGE=
LC_CTYPE=C.UTF-8
LC_ALL=
</code>```

What's going on?

答:
ENV set during the docker build will not be available to containers
In-order to persist ENV for  future containers you need to use ARG command.

So while build the image you need to pass ARG value which interns pass to ENV value and it will set to containers.

In docker file add this

ARG buildtime_variable=default_value.

ENV env_var_name=$buildtime_variable

And run this command

docker build --build-arg buildtime_variable=a_value # [...]

For detailed info follow below link
https://vsupalov.com/docker-build-time-env-values/

Body:
I've got this Dockerfile:

FROM ubuntu:latest

RUN apt-get update && apt-get install -y \
        curl \
        python3-dev \
        python3-setuptools \
        python3-pip

RUN rm -rf /var/lib/apt/lists/* && \
        apt-get clean

ADD . /usr/src/index-server
WORKDIR /usr/src/index-server

RUN pip3 install --upgrade pip
RUN pip3 install -r requirements.txt

EXPOSE 8000

ENV LC_CTYPE=C.UTF-8
ENV FLASK_APP=app_server.py

CMD /usr/local/bin/flask run --host=0.0.0.0

It builds fine. But when I try to run it, I run into a Python error, which I tracked to incorrectly set locale values.

On my host machine locale | grep -E 'LC_ALL|LC_CTYPE|LANG' returns:

LANG="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_ALL=

But when I try the same with the docker instance (docker run -t 657c402ec253 locale | grep -E 'LC_ALL|LC_CTYPE|LANG'), I get this:

LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_ALL=

If I use docker's -e flag to set the env variable, it works fine:

$ docker run -e LC_CTYPE=C.UTF-8 -t 657c402ec253 locale | grep -E 'LC_ALL|LC_CTYPE|LANG'

LANG=
LANGUAGE=
LC_CTYPE=C.UTF-8
LC_ALL=

What's going on?


问:
Does a procedural macro compile in one thread?
I have implemented a procedural macro to generate a method which returns a number:

```<code>extern crate unique_type_id;
#[macro_use]
extern crate unique_type_id_derive;

#[test]
fn sequential_simple() {
    use unique_type_id::SequentialTypeId;
    #[derive(SequentialTypeId)]
    struct Test1;
    #[derive(SequentialTypeId)]
    struct Test2;

    assert_eq!(Test1::id().0, 0u64);
    assert_eq!(Test2::id().0, 1u64);
}
</code>```

I have this implementation but I don't know whether should I place my static ID under a mutex or not:

```<code>fn inc_id() -> u64 {
    unsafe {
        static mut ID: u64 = 0u64;

        let old_value = ID;
        ID += 1;
        old_value
    }
}

fn sequential_implementor(ast: &syn::DeriveInput) -> quote::Tokens {
    let name = &ast.ident;
    let (impl_generics, ty_generics, where_clause) = ast.generics.split_for_impl();
    let id = inc_id();

    quote! {
        impl #impl_generics unique_type_id::SequentialTypeId for #name #ty_generics #where_clause {
            fn id() -> unique_type_id::TypeId {
                unique_type_id::TypeId(#id)
            }
        }
    }
}
</code>```

If the procedural macro only uses one thread then there is no need for a mutex, but I don't know this.

答:
And what happens when you use this in multiple files or even multiple crates? I think the compiler doesn't guarantee anything, and you simply shouldn't use global state at all (it might fork for proc_macro calls, run multiple compiler instances, use threads, ...).

To make it thread safe (and to avoid using unsafe) you could use AtomicUsize like this:

```<code>fn inc_id() -> usize {
    use std::sync::atomic::{AtomicUsize, Ordering, ATOMIC_USIZE_INIT};
    static ID: AtomicUsize = ATOMIC_USIZE_INIT;
    ID.fetch_add(1, Ordering::SeqCst)
}
</code>```

Body:
I have implemented a procedural macro to generate a method which returns a number:

extern crate unique_type_id;
#[macro_use]
extern crate unique_type_id_derive;

#[test]
fn sequential_simple() {
    use unique_type_id::SequentialTypeId;
    #[derive(SequentialTypeId)]
    struct Test1;
    #[derive(SequentialTypeId)]
    struct Test2;

    assert_eq!(Test1::id().0, 0u64);
    assert_eq!(Test2::id().0, 1u64);
}

I have this implementation but I don't know whether should I place my static ID under a mutex or not:

fn inc_id() -> u64 {
    unsafe {
        static mut ID: u64 = 0u64;

        let old_value = ID;
        ID += 1;
        old_value
    }
}

fn sequential_implementor(ast: &syn::DeriveInput) -> quote::Tokens {
    let name = &ast.ident;
    let (impl_generics, ty_generics, where_clause) = ast.generics.split_for_impl();
    let id = inc_id();

    quote! {
        impl #impl_generics unique_type_id::SequentialTypeId for #name #ty_generics #where_clause {
            fn id() -> unique_type_id::TypeId {
                unique_type_id::TypeId(#id)
            }
        }
    }
}

If the procedural macro only uses one thread then there is no need for a mutex, but I don't know this.

Body:
And what happens when you use this in multiple files or even multiple crates? I think the compiler doesn't guarantee anything, and you simply shouldn't use global state at all (it might fork for proc_macro calls, run multiple compiler instances, use threads, ...).

To make it thread safe (and to avoid using unsafe) you could use AtomicUsize like this:

fn inc_id() -> usize {
    use std::sync::atomic::{AtomicUsize, Ordering, ATOMIC_USIZE_INIT};
    static ID: AtomicUsize = ATOMIC_USIZE_INIT;
    ID.fetch_add(1, Ordering::SeqCst)
}


问:
Problems with the rails console, RVM and readline
I've installed rvm as a way of making sure that my local development version of ruby is the same as my server's for a particular app i work on (ruby 1.8.7).  I've done this, and installed ruby 1.8.7 ok.  However, when i try to start the rails console i get this error:

```<code>Readline was unable to be required, if you need completion or history install readline then reinstall the ruby.
You may follow 'rvm notes' for dependencies and/or read the docs page http://rvm.beginrescueend.com/packages/readline/ . Be sure you 'rvm remove X ; rvm install X' to re-compile your ruby with readline support after obtaining the readline libraries.
Couldn't load Wirble: no such file to load -- wirble
</code>```

I've read the notes on the page that error refers to (http://rvm.beginrescueend.com/packages/readline/), and followed the instructions, which involve basically installing readline, uninstalling ruby 1.8.7, then installing ruby 1.8.7 again with readline support.  (actually that page uses ruby 1.9.2 as an example but i'm assuming it should work with 1.8.7 as well.  Perhaps that's not the case).

But, i still get the same error.  Has anyone else been through this and figured it out?

Grateful for any advice - max

EDIT - i'm using Ubuntu 9.10 btw in case that's relevant.

EDIT - here's my output for rvm info as asked for below:

```<code>$ rvm info

system:

  system:
    uname:       "Linux max-laptop 2.6.31-22-generic #68-Ubuntu SMP Tue Oct 26 16:38:35 UTC 2010 i686 GNU/Linux"
    bash:        "/bin/bash => GNU bash, version 4.0.33(1)-release (i486-pc-linux-gnu)"
    zsh:         " => not installed"

  rvm:
    version:      "rvm 1.1.6 by Wayne E. Seguin (wayneeseguin@gmail.com) [http://rvm.beginrescueend.com/]"

  homes:
    gem:          "not set"
    ruby:         "not set"

  binaries:
    ruby:         "/usr/bin/ruby"
    irb:          "/usr/bin/irb"
    gem:          "/usr/bin/gem"
    rake:         "/usr/bin/rake"

  environment:
    PATH:         "/home/max/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/lib/gems/1.8/bin:/home/max/software/svnmerge:/home/max/.gem/ruby/1.8/bin:/home/max/work/e_learning_resource/trunk/public/assets/dvd_files/bin:/opt/firefox:/home/max/.rvm/bin:/home/max/.rvm/bin"
    GEM_HOME:     ""
    GEM_PATH:     ""
    MY_RUBY_HOME: ""
    IRBRC:        ""
    RUBYOPT:      ""
    gemset:       ""
</code>```

答:
I had a similar problem it was with 1.9.2 on Ubuntu 10.04, but the symptoms were identical.  In order to get it to work:

```<code>rvm pkg install readline
</code>```

or prior to RVM version 1.6.32, you need the following command

```<code>rvm package install readline
</code>```

Then:

```<code>apt-get install ncurses-dev
</code>```

And then taking from their instructions at http://rvm.beginrescueend.com/packages/readline/

```<code>cd $HOME/.rvm/src/ruby-1.9.2-p0/ext/readline
ruby extconf.rb -- --with-readline-dir="$HOME/.rvm/usr"
make install
</code>```

I had actually done a system rvm install on the server, so I needed to do this:

```<code>cd $HOME/.rvm/src/ruby-1.9.2-p0/ext/readline
ruby extconf.rb -- --with-readline-dir="/usr/local/rvm/usr"
make install
</code>```

After that I was able to have full access to the console.

Body:
I've installed rvm as a way of making sure that my local development version of ruby is the same as my server's for a particular app i work on (ruby 1.8.7).  I've done this, and installed ruby 1.8.7 ok.  However, when i try to start the rails console i get this error:

Readline was unable to be required, if you need completion or history install readline then reinstall the ruby.
You may follow 'rvm notes' for dependencies and/or read the docs page http://rvm.beginrescueend.com/packages/readline/ . Be sure you 'rvm remove X ; rvm install X' to re-compile your ruby with readline support after obtaining the readline libraries.
Couldn't load Wirble: no such file to load -- wirble

I've read the notes on the page that error refers to (http://rvm.beginrescueend.com/packages/readline/), and followed the instructions, which involve basically installing readline, uninstalling ruby 1.8.7, then installing ruby 1.8.7 again with readline support.  (actually that page uses ruby 1.9.2 as an example but i'm assuming it should work with 1.8.7 as well.  Perhaps that's not the case).

But, i still get the same error.  Has anyone else been through this and figured it out?

Grateful for any advice - max

EDIT - i'm using Ubuntu 9.10 btw in case that's relevant.

EDIT - here's my output for rvm info as asked for below:

$ rvm info

system:

  system:
    uname:       "Linux max-laptop 2.6.31-22-generic #68-Ubuntu SMP Tue Oct 26 16:38:35 UTC 2010 i686 GNU/Linux"
    bash:        "/bin/bash => GNU bash, version 4.0.33(1)-release (i486-pc-linux-gnu)"
    zsh:         " => not installed"

  rvm:
    version:      "rvm 1.1.6 by Wayne E. Seguin (wayneeseguin@gmail.com) [http://rvm.beginrescueend.com/]"

  homes:
    gem:          "not set"
    ruby:         "not set"

  binaries:
    ruby:         "/usr/bin/ruby"
    irb:          "/usr/bin/irb"
    gem:          "/usr/bin/gem"
    rake:         "/usr/bin/rake"

  environment:
    PATH:         "/home/max/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/lib/gems/1.8/bin:/home/max/software/svnmerge:/home/max/.gem/ruby/1.8/bin:/home/max/work/e_learning_resource/trunk/public/assets/dvd_files/bin:/opt/firefox:/home/max/.rvm/bin:/home/max/.rvm/bin"
    GEM_HOME:     ""
    GEM_PATH:     ""
    MY_RUBY_HOME: ""
    IRBRC:        ""
    RUBYOPT:      ""
    gemset:       ""

Body:
I had a similar problem it was with 1.9.2 on Ubuntu 10.04, but the symptoms were identical.  In order to get it to work:

rvm pkg install readline

or prior to RVM version 1.6.32, you need the following command

rvm package install readline

Then:

apt-get install ncurses-dev

And then taking from their instructions at http://rvm.beginrescueend.com/packages/readline/

cd $HOME/.rvm/src/ruby-1.9.2-p0/ext/readline
ruby extconf.rb -- --with-readline-dir="$HOME/.rvm/usr"
make install

I had actually done a system rvm install on the server, so I needed to do this:

cd $HOME/.rvm/src/ruby-1.9.2-p0/ext/readline
ruby extconf.rb -- --with-readline-dir="/usr/local/rvm/usr"
make install

After that I was able to have full access to the console.