Abstract Abstract-Smart toys have captured an increasing share of the toy market, and are growing ubiquitous in households with children. Smart toys are a subset of Internet of Things (IoT) devices, containing sensors, actuators, and/or artificial intelligence capabilities. They frequently have internet connectivity, directly or indirectly through companion apps, and collect information about their users and environments. Recent studies have found security flaws in many smart toys that have led to serious privacy leaks, or allowed tracking a child's physical location. Some well-publicized discoveries of this nature have prompted actions from governments around the world to ban some of these toys. Compared to other IoT devices, smart toys pose unique risks because of their easily-vulnerable user base, and our work is intended to define these risks and assess a subset of toys against them. We provide a classification of threats specific to smart toys in order to unite and complement existing adhoc analyses, and help comprehensive evaluation of other smart toys. Our vulnerability taxonomy addresses the potential security and privacy flaws that can lead to leakage of private information or allow an adversary to control the toy to lure, harm, or distress a child. Using this taxonomy, we perform a thorough experimental analysis of eleven smart toys and their companion apps. Our systematic analysis has uncovered that several current toys still expose children to multiple threats for attackers with physical, nearby, or remote access to the toy. INTRODUCTION W ITH smart toys set to grow to an $18 billion market by 2023 (as estimated by Juniper Research), we set out to classify the security and privacy threats they pose. Smart toys, which have sensors and/or actuators and internet connectivity, are in effect a subset of IoT. While threats posed by IoT have been extensively studied, we argue that smart toys present a distinctive attack surface because of their unique target user group (children and teenagers). A growing body of research shows that children harbor trust and attachment for smart toys [12] , [62] , [63] , [65] , and the effects of bugs and exploits in these toys are significantly amplified due to such a vulnerable user base. INTRODUCTION A child's trust for her/his toy can be exploited in several ways. A beloved toy may accompany a child wherever she goes, allowing comprehensive location tracking, e.g., via GPS or IP geolocation [33] . Bluetooth or WiFi enabled toys may broadcast a static MAC address, further facilitating location tracking. Children are inclined to trust their toys and follow their lead [55] , [61] . Our tests on the Sphero BB-8 showed that an unauthorized person could control the movement of the toy, potentially leading a child to follow the toy into danger. Using Wiggy Piggy Bank, an unauthorized person could issue rogue instructions to the child, for instance directing her outside the home. In both scenarios, the adversary could be within the Bluetooth range, about 40 meters. Finally, children, who tend to have wider trust boundaries than adults, are more inclined to share confidential and identifying information with a toy, and less likely to understand how that information will be shared and the consequences of sharing [30] , [33] . Some toys (e.g., Hello Barbie) actively solicit personal information, but many toys (including all the toys we examined) passively gather information, e.g., to infer usage patterns or improve personalization. Toys that require creation of an account online may collect additional information, e.g., email address. Some toys are equipped with microphone or cameras to capture audio, photo, and video-troubling enough if leaked, but in the worst case scenario pose the risk of being turned into an espionage device if an adversary can turn them on at will [8] . INTRODUCTION As a vulnerable population, children are at risk of being harmed by smart toy exploits. A child may be psychologically scarred if a trusted toy suddenly starts presenting disturbing, violent, or pornographic content. For instance, some of the toys we examined present content fetched over an unencrypted channel, making them subject to being maliciously replaced by a network attacker. A compromised toy with a speaker can be made to voice distressing audio [60] . Finally, a child can be harmed by any leak of information captured by a smart toy. The harm may be psychological; for instance, a child may experience a trusted toy revealing information thought to be confidential as a personal betrayal [29] . More saliently, it can result in immediate personal danger, e.g., if it reveals the child's location to an attacker, whether directly or indirectly, while others, like the disclosure of a child's birthday (requested by some of the toys we examined), can facilitate identity theft, which may become apparent only much later when the child reaches adulthood and the consequences are more difficult to undo [35] , [40] . INTRODUCTION Smart toys also share the vulnerabilities inherent to other arXiv:1809.05556v2 [cs.CR] 25 Oct 2018 IoT devices. Among these are the use of companion apps and third party libraries, increasing a toy's attack surface. An insecure or over-privileged companion app may be exploited to access the smartphone's microphone, camera, or GPS [37] , or directly control the toy; any communication between the toy and the app, if unencrypted, can also be intercepted or modified. Toy makers may store personal data insecurely or for an indefinite duration, increasing the likelihood of a breach, as does the use of third-party ads and analytics libraries [22] . Smart toy makers are also often simply the makers of other general purpose toys, with no particular expertise in information technology, particularly information security. They may thus not fully understand potential security risks to address them throughout the product life cycle [45] . This includes development, design, and testing phases, as well as after-market care, e.g., secure firmware updates should a vulnerability be discovered [58] . Several classification schemes have been proposed for general IoT security [1] , [11] , [26] , [51] , [71] , however nothing specific to smart toys. Some systematic analyses have been conducted into compliance of children's apps with children's privacy laws [31] , [49] , and experimental work on selected smart toys specifically [7] . Some inroads have been made into developing privacy paradigms for children's online activity and even into smart toys [32] , [46] , [70] . However, no taxonomy has been proposed that systematically addresses both the security and privacy of smart toys, specifically, for experimental evaluation. In this work, we examine eleven Internet-connected toys on the market and determine what Personally Identifiable Information (PII) they collect, what PII is shared with third parties like ads and analytics servers, what steps toy makers take to secure this data, and if the toy can be compromised in a way that poses a threat to the physical or psychological safety of the child using it. We also define a list of attacks categorized by attacker proximity and assess the smart toys against them. arXiv:1809.05556v2 [cs.CR] 25 Oct 2018 Traditional threat taxonomies devised for general IoT devices do not, in our opinion, adequately address the threats they pose to children specifically. Some threats we present are specific to smart toys (e.g. parental PII protection), while others are given more emphasis (e.g. exposure to third party ads and analytics services) compared to existing taxonomies. Regulatory frameworks, like COPPA in the US, that address children's digital privacy specifically recognize that children face unique harms from such devices, and are designed to protect children against these threats. arXiv:1809.05556v2 [cs.CR] 25 Oct 2018 We consider two broad categories of risk: breach of PII, and unauthorized control of the toy. While there is no universal definition of PII [52] , we have closely adhered to that specified in COPPA, as the most widely applicable regulation to specifically address children's privacy [57] . We have categorized PII into the following: personal information including name, gender, physical address, email address, telephone number, voice recordings, and photos; device information, including mobile device unique identifiers including IMEI, serial number, MAC address, IP address, and smart toy ID; and service usage information, including session start and end time, session duration, and app features. Our findings show that popular smart toys are vulnerable to several attacks in our taxonomy, some of them severely. For example, in Toymail, we found that blocked contacts still receive children's personal information. Wiggy Piggy Bank allows nearby adversaries to issue arbitrary tasks to a child, in the familiar voice of the toy. Smart Toy Monkey sends personal information unencrypted over HTTP to a server located in China. VTech InnoTab MAX does not require any authentication to access a child's personal information. Contributions. 1) We develop an experimental framework for evaluating security and privacy of smart toys. The framework encompasses PII collected and transmitted by smart toys, and security measures that have been taken to protect them; and PII collected and transmitted to third parties such as ads and analytics services, and the third parties' TLS practices. Our framework relies on analysis of network traffic (WiFi and Bluetooth), and reverse engineering and code analysis of companion apps. 2) We investigate a representative, diverse set of eleven smart toys using our experimental setup to expose potential PII leakage, weak security measures, and other vulnerabilities. Our methodology and tests can be directly used for other toys on the market, and extended/adapted for evaluating future toys. 3) We uncover, through experimentation, an excessive collection of unique IDs that facilitate tracking children across different services or platforms, and sending children's PII to unauthorized entities. Our results also show that several toys expose children to threats that may be exploited by physical, nearby, or remote attackers. Contributions. As part of responsible disclosure, we have contacted the manufacturers of all toys mentioned in this paper and shared our findings with them. Wiggy Piggy Bank responded quickly and mentioned that they are looking into the vulnerabilities and possible fixes. Hello Barbie responded that their technical team may get back to us "if they have any concerns." Anki Cozmo discarded our findings, describing them as "unsolicited ideas." We have also received automatic replies from Osmo, Cloudpets, Sphero BB-8, and Toymail; the rest have not responded even after 6 months. Overall, most responses (and lack thereof) indicate that these companies take privacy matters much more frivolously than expected. RELATED WORK Smart electronics for children have made the news for security breaches numerous times in the past; e.g., see Rapid7 [47] (strangers could hack baby monitors to view children sleeping and even talk to them directly), Norwegian Consumer Council [16] (highlight dangerous security vulnerabilities in 3 toys: i-Que, My Friend Cayla, and Hello Barbie), Motherboard [17] (strangers could track children's location and message them through the HereO smart watch), and BBC [4] (German ban on Cayla in February 2017, branding it as an "illegal espionage apparatus"). Like wearables for adults, many smart toys are Bluetoothenabled and accompany children outside the home, and previous research on fitness trackers has uncovered numerous vulnerabilities such as location tracking [23] . RELATED WORK Several security breaches involving the VTech InnoTab MAX have been reported. In 2015, VTech's Learning Lodge database, used by the VTech suite of children's tablets, suffered a severe breach, exposing the PII of 6.3 million children's profiles [27] . The breach was due to a combination of insecure practices, e.g., PII was sent in plaintext (no HTTPS), the database was vulnerable to a simple SQL injection attack, and account passwords were stored using an MD5 hash (no salting). The breach ultimately led to a $650,000 settlement with the U.S. Federal Trade Commission for unauthorized collection of children's personal information and for failing to secure that information [59] , and a related finding from Canada's Privacy Commissioner that VTech was in breach of its privacy laws (but no corresponding settlement) [43] . Also in 2016, the UK-based security firm Pen Test Partners found that the VTech InnoTab MAX is vulnerable to trivial data extraction [44] . (We found that this flaw has not been fixed.) Later in 2016, it was found that pornography could be easily accessed through the built-in browser by using Google Translate, formerly white-listed but since removed by VTech [24] . Rapid7 also found severe vulnerabilities in the Fisher Price Smart Monkey, since fixed [48] . RELATED WORK Hello Barbie has been extensively scrutinized for its privacy and security practices [14] , [25] , [38] . It encourages children to divulge intimate details, which are then shared with Mattel's partner, ToyTalk, and with parents through the web portal [29] , [54] , presenting legal implications if a child discloses any physical or sexual abuse to the toy [34] . In January 2016, Somerset Recon Inc. RELATED WORK [53] reported several vulnerabilities in Hello Barbie, some of which are still valid as we found (e.g., broadcasting an open hotspot and allowing unauthorized configuration during pairing). Popular brands of children's smart watches have been analyzed by the Norwegian Consumer Council, highlighting vulnerabilities in all watches they examined [42] . In November 2017, "Which? UK" issued a report examining common Bluetooth vulnerabilities in children's toys that allowed them to be taken over by nearby attackers [64] . Given the stakes at risk, the US Senate tabled a report on smart toy security in December 2016 [39] . The following July, the FBI issued an alert warning parents against privacy and security concerns regarding smart toys [56] . RELATED WORK Rafferty et al. [46] argue that the traditional access control model is insufficient to protect the privacy of children using smart toys. They introduce a conceptual model for smart toys, allowing parents to configure privacy rules and receive notifications about sensitive data disclosure. The model assumes that children are oblivious to privacy concerns, and as a result, cannot adequately protect themselves online or anticipate the consequences of leaking private information to smart toy makers and other third parties. RELATED WORK Mahmoud et al. [32] propose an analytical framework with 17 privacy-sensitive criteria to evaluate smart toy privacy practices. They analyzed the available privacy policies and terms-of-use documents for 11 smart toys, and reported potentially excessive collection of children's private information and lack of information on data storage location and legal compliance. They also performed static analysis of companion apps of these toys, and found most apps to be over-privileged with the potential to leak children's private information. Our taxonomy, in contrast, offers a classification of smart toy-specific vulnerabilities. We further investigate the actual (as opposed to the potential) security and privacy practices of a similar set of smart toys through dynamic analysis by using the toys and apps in a realistic scenario; our experiments measure PII collected by the toy manufacturers and third-party servers, and uncover concrete security practices and vulnerabilities. RELATED WORK In summary, past work mostly involved isolated case studies exposing vulnerabilities in an individual smart toy, or analytical frameworks for smart toy privacy practices. In this work we propose a taxonomy of vulnerabilities pertinent to smart toys, categorize them by attacker proximity, and comprehensively assess a set of toys against it. EXISTING REGULATORY FRAMEWORKS Internet-connected toys must comply with traditional toy safety regulations, and as online services, they must adhere to applicable digital privacy laws. Where there are such laws, they tend to be more restrictive with respect to children, in keeping with the common understanding of children requiring greater legal protection. In what follows, we briefly discuss EU and US laws and regulations, as we also evaluate the toys in terms of following relevant regulatory restrictions. EXISTING REGULATORY FRAMEWORKS In the EU, the General Data Protection Regulation (GDPR) governs how consent should be obtained when collecting children's personal information. Specifically, a consent must be obtained by a parent or guardian for children under the age of consent (13-16 years, varies among the member states), and privacy notices should be written in clear, age appropriate language [10] . Data collected from children is otherwise subject to the same rules that apply to any other EU citizen, including data transparency (the ability of an individual to determine what data has been collected about them), data erasure (the right to erase that data, otherwise known as the right to be forgotten), and data portability (the ability to obtain the data for reuse across online services). The GDPR also mandates how companies must respond in the event of a breach, and penalties may be imposed for failing to report the breach to the applicable authority and notifying the affected individuals in a timely fashion. EU citizens do not have to reside in the EU to be protected by the GDPR, even if they use services from a company without any legal presence in the EU market [9] . EXISTING REGULATORY FRAMEWORKS In the US, the Children's Online Privacy Protection Act (COPPA) governs data collection and online tracking of children under the age of 13. Online services should take reasonable measures to obtain parental consent to collect and disclose information, clearly outline data retention policies, and provide a means for data review and deletion. Under COPPA, private information is defined to include persistent identifiers like an IP address, which by default is included in all TCP/IP traffic; thus, by definition, all internet-connected toys collect private information. MAC addresses, like those of a toy's Bluetooth or WiFi interface, are also considered to be persistent identifiers; in our tests, we have observed MAC addresses transmitted to first party toy servers as well as third-party analytics servers. However, merely collecting persistent identifiers (absent parental consent) is not in and of itself a violation of COPPA, if it is not used in conjunction with any other identifying information, or if it is not stored or made public, or if it not used for any other purpose than to ensure proper operation of the service (in this case, the toy). If the toy or companion app contacts a third-party analytics company, which then uses persistent identifiers to build a profile of the child, or passively track the child online, it is a COPPA violation, absent explicit parental consent. Further, the toy's privacy policy must explicitly list all such third party companies [13] . EXISTING REGULATORY FRAMEWORKS Prior work has analyzed toys' privacy policies for compliance with these laws, and compared, through static analysis, the toys' stated policies with the privileges requested by their companion apps. Through dynamic analysis, we have found excessive collection of persistent identifiers by toy makers and ads and analytics services, and in some cases, we also have found a clear violation of COPPAs injunction against persistent tracking for the purpose of advertising. In addition, while both COPPA and GDPR do not explicitly address issues of security, we have found security vulnerabilities that expose PII to unauthorized parties (e.g. unauthorized configuration and use), which violates the spirit of such regulations if not their actual content. ANALYSIS FRAMEWORK Most of the smart toys we analyze are intended to be used with companion apps running on smartphones or tablets. All the mobile apps we examine, and all toys with direct connectivity, communicate with remote hosts. This presents a wide attack surface covering toys, apps, remote hosts, and the communication between each pair of these; see Fig. 1 . Security of PII collection and transmission Toys that require the user to create an account typically collect and store more PII than toys that do not. We audit the network traffic of smart toys and their companion apps to determine the PII they collect, transmit, and share with toy servers and third parties, and whether those third parties are ads or analytics servers. Security of PII collection and transmission We evaluate the security measures for protecting the collected PII in local storage and when being sent to a remote server. In some cases, we also verify that non-PII or non-confidential content such as toy name and images (in toy/web UI) are protected from malicious tampering. Data transmission (including PII) can occur between the toy and a companion app, a companion app and a remote server, or, for toys with Internet connectivity, the toy itself and a remote server. Additionally, data may be stored locally on the toy or on the device hosting the companion app. To investigate the confidentiality and integrity of data, we assess the security of communications channels between each pair of endpoints and local data storage mechanisms. Security of PII collection and transmission We determine the protocols used to communicate with remote hosts (e.g., HTTP or HTTPS), and assess the likelihood and severity of active man-in-the-middle attacks, from replacing UI elements in the app with inappropriate content to remotely controlling the toy. Potential vulnerabilities We define a list of potential vulnerabilities and assess the smart toys against them. These vulnerabilities are grouped by the proximity required to exploit them. Physical access. The following vulnerabilities can be exploited by an adversary with physical access to the toy or companion smart device. No-local-PII-protection. An adversary can retrieve PII stored locally in the toy's internal storage, or within the companion app, leaving it unprotected in the event the toy/device is lost, stolen, or sold unsanitized. Unauthorized-config-physical. An adversary can configure the toy/device to maliciously forward PII to their account, or issue harmful commands to the child. Nearby access. These vulnerabilities require an attacker to be near the toy (e.g., within WiFi/Bluetooth range). Unauthorized-use. This vulnerability allows an attacker to use the toy, for instance to remotely control it. Unauthorized-config-nearby. An attacker can download the companion app in his own device, then connect to the toy, and maliciously configure it. Unencrypted-hotspot. Some toys can toggle their WiFi adapter into access point mode to allow a companion app to connect directly to the toy and configure it. A toy's hotspot should authenticate devices that attempt to connect, and use an encrypted channel. Otherwise, an adversary who connects to a toy's open hotspot can launch an MitM attack and sniff PII in plaintext, or maliciously configure the toy. Insecure-Bluetooth-practice. The use of static Bluetooth MAC addresses allows a toy, and the child using it, to be tracked persistently (cf. [23] ). Also, accepting unauthorized Bluetooth connections by the toy allows adversaries to connect to the toy and change its behavior, or launch an MitM attack to sniff information transmitted between the toy and the app (cf. [6] ). Bluetooth Low Energy (BLE) resolves these two problems by MAC address randomization and whitelisting [15] . Always-on. Toys that do not have power switches and include sensitive sensors or are connected to the Internet, increase their exposure to potential attack, for instance by continuously broadcasting a static MAC address. Remote access. The following vulnerabilities expose the toy to remote attacks. Online-password-bruteforce. Some toys allow Web login to the parent account. Login pages that allow unlimited number of password trials could allow an adversary to brute force passwords, particularly if they are weak. Potential vulnerabilities No-remote-PII-protection. This vulnerability allows adversaries to access PII remotely without proper access control (e.g., allowing access to audio/video recordings through a link, without any further authentication). Weak-parental-PII-control. Some toys do not allow parents to review or delete PII collected from their children, such as voice recordings or pictures. Exposure-to-third-parties. Third parties with whom PII is shared, such as ads and anaytics server, may have inadequate measures to protect collected data. Unencrypted-comm-channels. Information exchange between different parties (toy, app, and hosts) may be susceptible to interception through a MitM attack (e.g., if HTTP is used). Insecure-session-cookies. Session cookies are used to automatically log users in to their accounts. If they are not adequately secured, for instance if they do not expire or the secure flag is not set, an adversary may gain unauthorized access to the parent account and to the child's profile. Insecure-TLS-practices. TLS vulnerabilities [66] may result from using weak cipher suites, old TLS versions such as SSL 3.0, and vulnerable extensions. We examine servers contacted by the toy against protocol vulnerabilities such as POODLE, CRIME, Heartbleed, and Ticketbleed [50] , and assess server certificates for security issues, including using weak cryptographic keys and certificate mismatch. URL-redirect. Web servers that are not hardened against a URL redirect vulnerability could allow an adversary using social engineering techniques to redirect users to phishing websites (e.g., to steal their credentials). An adversary could send links to users that appear legitimate (e.g., containing the correct domain name), but that use special characters to redirect the user to a malicious domain. Servers should prevent URL redirect, or at least whitelist accepted URLs. It should be noted that an ideal taxonomy of threats would include other common web-based vulnerabilities, such as those in the OWASP top-ten most critical web application security risks. We have not presented these here as they require more intrusive probing of the web sites in question (e.g., cross-site scripting (XSS)), which for legal reasons, require consent from the companies in question. METHODOLOGY AND EXPERIMENTAL SETUP Using our threat classification model, we perform case studies on eleven well-known smart toys, described in Appendix A. We systematically assess the toys, their companion apps, and the remote servers they communicate with against our model. To obtain data for our analysis, we systematically run the toys through typical use case scenarios, using real mobile devices to circumvent evasion techniques apps might use to avoid detection of suspicious activity. Existing app analysis tools are used to detect the servers the toys communicate with, the PII they transmit, and the security measures applied by toy makers to protect personal information. We augment these wherever suitable by applying reverse engineering mechanisms, network traffic analysis, retrieval of certificate private keys, and leaking protection passphrases. Experimental setup We maintain two experimental setups. Our first environment uses a PC hosting Windows 10 professional 64-bit. The PC is configured to use a MediaTek 802.11G WiFi adapter in access point mode, using the 2.4 GHz frequency band (the band used by the toys we examine), and a second NIC with Internet connectivity. We examine the companion apps using a Samsung N7100 hosting Android 4.4.2. This version of Android is ranked among the top four Android versions used in the wild [20] . As an older version of Android, it is illustrative of how toys deal with older cipher suites, TLS versions and TLS protocol vulnerabilities. Our second environment uses a MacBook tethered via USB to an iPhone providing Internet connectivity and configured to act as an access point. Companion apps ran on iOS 10, both on an iPad Air and an iPhone 7. We chose iOS 10 as it was the latest iOS version at the time we began experimentation in June 2017, and statistics show that the majority of iOS users run the latest version of the OS [41] . We tested one group of toys on an Android platform (Toymail, Wiggy, Barbie, CHiP, and Cloudpets) and the other group on iOS (Sphero, Cozmo, Ozobot, Osmo, and Monkey). We hope to expand on this work in the future by performing a systematic comparison of the latest versions of both platforms. Network analysis Both experimental setups use Wireshark 1 to sniff network traffic initiated by the toys and their companion apps. To decrypt traffic, our first setup uses Burp Suite 2 as a MitM proxy and adds the Burp Suite CA certificate to the smartphone CA store. Our second setup uses MitMProxy 3 in lieu of Burp Suite, and correspondingly installs MitMgenerated certificates on the devices running the apps. We address special cases where this approach was insufficient to intercept and decrypt TLS communications. All such interventions were performed on Android-based client apps that were reverse-engineered. The Qualys SSL Labs 4 testing suite was used to perform TLS analysis on the server side, and we examined packet captures in Wireshark to analyze TLS practices on the client app. Ads and analytics analysis Wherever possible, we intercept data between the companion app and third party ads and analytics servers (Internetcapable toys we examined do not directly communicate with ads and analytics services). To determine the extent to which this data can be used to uniquely identify the user across multiple apps on the same device or across different platforms on different devices, we created an account on a selected analytics server, Flurry Analytics, used by two of our toys, and wrote and deployed a dummy app to connect to it. Using this method, we were able to determine that Flurry Analytics used the same unique identifier on both a companion app and our dummy app, despite the fact that the vendor ID is different on both. This identifier can be used to distinguish the user across unrelated services on the same device. Other analytics servers we examined correlate unique identifiers with users' personal information such as email address, or use cross-platform cookies such as DSID and IDE [19] cookies to distinguish users across different platforms. App analysis All companion apps were analyzed manually (as opposed to using, for example, the Monkey automation tool 5 ). We mimicked common use case scenarios with the goal of triggering app UI events looking for signs of PII leakage, weak security measures, or potential vulnerabilities, and captured data from the interaction with the toy and its companion app. Modify custom CA store. When the companion app uses a custom CA store to verify server certificates, we apply the following methodology to force the app to accept the MitM certificate: (a) Decompile the app using Apktool 6 to retrieve the CA store from the app's assets directory, (b) Patch parts of the app smali files to force revealing the CA store password, (c) Use the password to access and update the custom CA store, (d) Use the keystore explorer tool to add the MitM certificate, and (e) Replace the CA store in the assets directory of the app with the new store, rebuild the app using Apktool, re-sign and verify the patched apk file, and use the adb tool to reinstall it. As a result, the app accepts the MitM certificate. Exfiltrate client SSL certificate. In cases where the companion app authenticates to the server using a client Bouncy Castle file (PKCS#12), which encompasses the client's public key and the client's private key, we reverse engineer the app to exfiltrate the PKCS#12 file and the passphrase used to protect it. We then add the PKCS#12 file (the certificate and the private key) to the interception proxy. This allows the interception proxy to authenticate to the server. Bypass certificate pinning. In cases where the companion app uses certificate pinning to refuse server certificates signed by any CA other than the one with the pinned certificate in the app, we patch the corresponding parts of the app smali files to disable this feature and intercept the communication. TLS vulnerabilities analysis We assess toys, apps and server SSL practice to determine potential TLS protocol vulnerabilities, and use Qualys SSL Labs to determine TLS server parameters, including supported TLS versions, cipher suites and extensions. Bluetooth analysis BLEscanner 7 is used to examine the Bluetooth connection between the toy and the companion app, and determine whether the toy's Bluetooth MAC address is persistent or dynamically changing. We investigate how an adversary could tamper with the communication or gain unauthorized access to Bluetooth parameters to conduct MitM or other attacks [28] . The BTLEjuice tool 8 attacks on toys that use Bluetooth to exchange data with a companion app, to both sniff and alter the communication, and RamBLE 9 is used to modify BLE settings. SUMMARY OF RESULTS We systematically applied our threat classification scheme to a sample of 11 popular smart toys on the market, varying by target age range and functionality. In particular, the toys we selected span multiple target ages, Nine of the eleven toys we examined are vulnerable to some form of attacks, whether through physical or nearby access (within WiFi or Bluetooth coverage), or remotely, such as over HTTP; see Table 1 . SUMMARY OF RESULTS In terms of PII collection and contacting third-party servers, all toys we examined collected PII, and 9 out of 11 communicated with one to four ads and analytics servers; see Table 2 . Alarmingly, the toys for younger children collected PII more aggressively and frequently, including intrusive PII, such as cross-service and cross-platform unique identifiers and user and devices fingerprinting information; these toys also in general protected PII more poorly. One such toy in our samples (Cloudpets) also displays ad in its companion app. In contrast, toys for older children collected less PII, generally did not require account creation, and had more thoughtful security measures in place. Table 3 shows the persistent user IDs collected by the toys and companion apps. Apparently, some IDs captured in the transmissions were obfuscated, or their purpose was unclear. In such cases, we assume that if they appear across multiple sessions, like an app ID, then they are deemed to track the user across multiple sessions of the same app. All the Android apps we tested (Toymail, Wiggy Piggy Bank, Hello Barbie, Wowwee Chip, and Cloudpets) transmit the Google advertising ID, even when ad tracking is disabled in the smartphone. These apps, and those that transmit a hardware ID, are deemed to track users across multiple services on the same device. Finally, apps that use cookies such as DSID and IDE on Android [19] , or transmit email addresses or other identifying information about the user to analytics servers, are deemed to track users across multiple platforms. On iOS apps, the smartphone's advertising identifier was not transmitted when the "Limit ad tracking" was disabled. Table 4 shows types of PII collected by first-party toy servers. Note that we exclude any PII stored locally on the toy or smartphone. Multimedia may consist of audio (e.g., voice recordings), photos, or video, which may be recorded by the user as part of toy functionality, or uploaded as 9 . https://www.contextis.com/resources/tools/ramble-ble-app 10. For example, Anki Cozmo is marketed as an AI toy: "Anki Cozmo, is an AI toy robot with a big brain and even bigger personality" (https://www.anki.com/en-ca/cozmo (in the page metadata). TABLE 1 Attacks by proximity Physical Nearby Remote N o -l o c a l -P I I -p r o t e c t i o n U n a u t h o r i z e d -c o n fi g -p h y s i c a l U n a u t h o r i z e d -u s e U n a u t h o r i z e d -c o n fi g -n e a r b y U n e n c r y p t e d -h o t s p o t I n s e c u r e -B l u e t o o t h -p r a c t i c e A l w a y s -o n O n l i n e -p a s s w o r d -b r u t e f o r c e N o -r e m o t e -P I I -p r o t e c t i o n W e a k -p a r e n t a l -P I I -c o n t r o l E x p o s u r e -t o -t h i r d -p a r t i e s U n e n c r y p t e d -c o m m -c h a n n e l s I n s e c u r e -s e s s i o n -c o o k i e s I n s e c u r e -T L S -p r a c t i c e s U R L -r e d i r e c t Toymail profile information. We consider a toy to transmit geolocation only at street-level granularity or below, not above (e.g., city). All the toys we examined transmit the user's public IP address, but in addition one (Toymail) sends the private IP address of the toy on the local network. Toymail is unique also in that it transmits the user's Facebook ID and the sleep/wake cycle of the child. RECOMMENDATIONS In this section, we list our recommendations for makers of connected toys, beyond following existing safety standards and privacy regulations. RECOMMENDATIONS Fixing vulnerabilities. Toys, like other more generalized IoT devices, should have a process for addressing vulnerabilities. This includes a way to disclose vulnerabilities, including: a formalized reporting mechanism, a simple link to a form on their web site, or subscription to a bug bounty program, like HackerOne (hackerone.com) or BugCrowd (bugcrowd.com). At the very least, vulnerability disclosure should not be discouraged by prohibiting under the terms of use. There should be a short turnaround between identifying vulnerabilities and fixing them in the firmware or app software where they are located. Finally, it is critical to be able to push out new firmware or software updates and patches as soon as they are available. There are existing mechanisms to update companion apps through the corresponding Android or iOS app store, but pushing out firmware updates to the toy is slightly more challenging. Toys with Wi-Fi connectivity should periodically check for updates, and companion apps should check at the launch time for available updates. However, relying on the user to manually launch the app extends the window of potential exploitation between discovery of the vulnerability and time it is patched, so it is preferable for apps to have push notification functionality enabled. For this reason, companion apps should request permission to push notifications on first launch, and clearly state why this can improve the security and safety of the toy. (It is also incumbent on the app to not abuse the permission with unnecessary notifications lest the user disable it.) RECOMMENDATIONS Limiting data collection, and improving storage and communication security. Fine-grained access control. Toy companies that provide links to PII on public servers, such as voice recordings, should also grant parents the ability to select whether the PII is public or private. Even when links are randomly generated, if they are sufficient to access PII without any additional authorization, it still can pose a privacy threat. We also recommend allowing the parent to configure whether such links are public or private, defaulting to private. Finally, these links should expire after a reasonably short period of time. RECOMMENDATIONS End-of-life. Toys should provide a graceful means of scrubbing identifying and personal data in the event of endof-life, loss, theft, or transitioning owners. This may mean restricting the amount of personal data stored locally on the toy to only what is strictly necessary, and requiring some form of authentication to access whatever data remains. Toys that require the creation of online accounts tend to collect more personal information than toys that do not, at least in our sample, but also provide a convenient interface for letting a parent log in through a browser and dissociate the toy from the account if necessary. This ensures that an unauthorized third party with access to the toy cannot access the related account and any personal data stored within it, or use it to communicate with the child. Advertisement and tracking. We strongly caution against ad display in companion apps, especially as these apps form part of a purchased toy and thus should not require support through ad revenue. In addition, ads and analytics services should be used sparingly with children [3] , and when used, should not collect any PII. Various other measures can be taken to protect the user's identity when using analytics services; Google Analytics, for instance, provides a means of anonymizing IP addresses [21] . Flurry Analytics prohibits the use of its services with children under the age of 13 (in accordance with COPPA) unless its restricted feature set limiting user profiling is used [69] . Apps should respect users' request to disable ad tracking. This is an issue particularly on Android platforms. Specifically, we observed multiple instances of Android apps flouting the user's preferred ad tracking setting, and sending the ad identifier regardless of whether it was disabled. By contrast, iOS app developers cannot access a valid ad identifier on a device that has ad tracking disabled [2] . In the Android model, a single system call returns both the ad identifier and the limit ad tracking setting, and the developer is assumed to not abuse them [18] . CONCLUSIONS In this paper, we examined the security and privacy practices of eleven popular smart toys on the market. Our findings show widespread use of data collection by toy makers and third party analytics servers, with almost all toys embedding analytics services within their apps, and many of them transmitting advertising identifiers even when ad tracking is disabled by the user. Several toys do not take adequate measures to protect sensitive data, and some toys do not protect locally stored PII at all, making it trivially accessible to anyone with physical access. Children's physical safety may also be at risk, as all Bluetooth-capable toys we examine advertise a static MAC address, making it possible to track them through physical space. Finally, several toys are vulnerable to exploits that can result in handing an attacker control of the toy or some of its functionality, potentially causing harm to a child. APPENDIX A RESULTS ANALYSIS OF SELECTED SMART TOYS In this appendix, we present individual case studies for the toys mentioned in the main body of the paper. A.1 Toymail Toymail allows children to exchange voice messages with parents, relatives and friends using the companion app or another Toymail. Parents can configure the toy with an approved list of contacts. Voice alerts notify children when they have new messages, and they can click the play button to listen. PII collected and transmitted. The companion app sends PII to app.toy mailco.com including login credentials; email address; parent profile ID; child name, date of birth, and photo; friend profile ID, name, and profile picture; and voice messages. The app communicates with multiple ads and analytics services which collect personal information as follows. E.crashlytics.com collects smartphone device information and app crash information, along with smartphone hardware ID, Google ads ID, and app installation ID. Api.branch.io collects smartphone device and OS information, Toymail user ID, smartphone hardware ID, smartphone fingerprint ID, IP address, identity ID, and email address. Api.branch.co verifies whether the smartphone is a real device or an emulator. Wzrkt.com sets a unique ID for the user which it collects while the app is running, allowing tracking the user across different sessions. It also collects the Google Ad ID (allowing tracking the user across different services), smartphone device information, and telephone service carrier information. Api.segment.io collects Toymail user ID, name, email address, device hardware ID, Google ads ID, smartphone device and OS information, data network device status (WiFi, Bluetooth, cellular), and phone carrier name. None of these ads and analytics servers respects the device-wide ad tracking setting, and all collect PII regardless of whether it was disabled. Unsurprisingly, we found that using the toy resulted in targeted ads even on different platforms by setting unique identifiers (e.g., identity ID) and correlating them with other personal information (e.g., email address), as shown in Fig. 2 which depicts a Toymail ad in a PC hosting Windows, on a different WiFi network. A.1 Toymail Once the toy connects to the WiFi network, it communicates with imp07a. boxen.electricimp.com to receive a list of allowed contacts. When the child selects a contact and presses the message button, the toy begins recording. When the child presses the message button again, the toy transmits the voice recording to the server, which stores them in Amazon AWS storage. The toy keeps up to ten voice messages locally which are received from parents and contacts. It also stores Toymail child ID, toy ID, contacts' names, and contacts' Toymail IDs, in addition to WiFi SSID and password. Security measures. The app connects to all servers over TLS. The app.toy mailco.com server certificate is signed by Go Daddy and uses a 2048-bit RSA key. The app maintains its own list of strong cipher suites, and uses certificate pinning to prevent MitM attacks. It stores hashes of the certificate chain public keys to bind the server certificate with the original issuer. However, the toy suffers from a number of Insecure-TLS-practices. The imp07a.boxen.electricimp.com server certificate is signed by the custom root CA impca, and both certificates are sent to the toy. The server certificate is valid for 20 years and it uses a 1024-bit RSA key whose certificate common name is different from the server's domain name. The toy uses a client certificate to authenticate to the server. It is signed by impca, uses a 2048-bit RSA key, and is valid for 100 years. The app communicates with two analytics and ads servers vulnerable to RC4 and POODLE attacks, with support for SSL 3.0 and weak cipher suites, exposing Toymail to Exposure-to-third-parties. Neither the session cookie expiration date nor secure flag are set, leaving the toy vulnerable to Insecure-session-cookies. A.1 Toymail Toymail is rated to offer Weak-parental-PII-control. When a child sends recordings to contacts other than the parent, they are stored in the contact's account, leaving the parent without means to review or delete them. Even when a parent blocks a contact from the child's contact list, that contact can still access the child's personal information. The server sends the child's personal information to the contact's app accompanied by a "blocked" flag (equal to "-1") indicating that the recipient has been blocked. The app removes the entry corresponding to the child from the app interface so they can no longer exchange voice messages; however, the contact still receives the child's personal information each time the app is launched including: name, date of birth, photo, voice recording of the name, parent Toymail ID, toy ID, wake up time, sleep time, and "isOnline" flag showing the state of the toy. A.1 Toymail With physical access to the toy, anyone can access these messages and even add contacts to the contact list without authentication, making it vulnerable to No-local-PIIprotection and Unauthorized-config-physical, respectively. In fact, there is no mechanism to delete locally stored voice recordings other than configuring the toy to use another account. A.1 Toymail Toymail is designed to be Always-on. Switching off requires removing the batteries, accessible only by removing the back cover using a screwdriver. Given that the toy includes sensitive devices like a microphone and is connected to the Internet, we deem this a significant potential threat to children's privacy and security. A.2 Wiggy Piggy Bank Wiggy Piggy Bank is a Bluetooth-connected toy that allows parents to set tasks, create goals, and send rewards to their children through a companion app. Children are notified they have new tasks either by the toy, which gives alerts and states the tasks, or by checking their accounts through the app. Children confirm through the app that they have completed a task, and upon parental approval virtual funds are transfered to their account. A child can set goals (e.g., buying a mobile phone for $100). When the balance of the child's virtual account achieves the target value of the goal, the child may redeem it from the parent. PII collected and transmitted. The app communicates with the server api.kii.com, sending parent login credentials (email address and password or session cookie), user ID, children's profile pictures, tasks set by parent, tasks completed by children, and children's goals. The app also communicates with ads and analytics services. Unity3d.com collects an app ID, device unique ID, user ID, session ID, and device and OS information. Moreover, even when ad tracking is disabled, unity3d.com still collects an advertising ID uniquely identifying the user across different services, and app usage information including app start time and duration. The app also connects to google-analytics.com, which collects analytics data and applies obfuscation techniques before sending them. Security measures. The app's connection to api.kii.com appears to use TLS in a secure way. The app relies on the OS trusted store to check the server certificate. The server uses a certificate signed by well-known certificate authority (Go Daddy Secure Certificate Authority -G2) and signed by a 2048-bit RSA key using SHA256. The server public key uses a 2048-bit RSA key and the certificate is valid for a limited period (expiring in October 2019). The server has six cipher suites that support forward secrecy and they are at the top of the server's cipher suites list. The server is patched against vulnerabilities including POODLE and Heartbleed, Ticketbleed, and does not support TLS compression, which causes vulnerabilities like CRIME. Although the app uses HTTPS to communicate with the server api.kii.com, the server does not force using HTTPS in the case of flipping HTTPS to HTTP, leaving all communication unencrypted, and making the toy vulnerable to Unencrypted-comm-channels. We use the Burp Suite repeater tool to flip all app requests to the server on the fly to HTTP instead of HTTPS, and as a result, the app connects to the HTTP version of the server and receives PII in plaintext. A.2 Wiggy
{'paper_id': '53960754', 'title': 'Playing With Danger: A Taxonomy and Evaluation of Threats to Smart Toys', 's2_url': 'https://api.semanticscholar.org/CorpusID:53960754', 'source': 'S2ORC'}