Question: Represent the text to find a very short summary (1-2 sentences max).

Question: We were approached by a group of healthcare providers who are involved in the care of chronic patients looking for potential technologies to facilitate the process of reviewing patient-generated data during clinical visits. Aiming at understanding the healthcare providers' attitudes towards reviewing patient-generated data, we conducted a focus group with a mixed group of healthcare providers. Next, to gain the patients' perspectives, we interviewed eight chronic patients, collected a sample of their data and designed a series of visualizations representing patient data we collected. Last, we sought feedback on the visualization designs from healthcare providers who requested this exploration. We found four factors shaping patient-generated data: data & context, patient's motivation, patient's time commitment, and patient's support circle. Informed by the of our studies, we discussed the importance of designing patient-generated visualizations for individuals by considering both patient and healthcare provider rather than designing with the purpose of generalization and provided guidelines for designing future patient-generated data visualizations. Collecting patient-generated data is becoming increasingly common in chronic disease management. Patients use technological tracking tools to collect health and lifestyle data in disparate places. Both healthcare providers and patients agree that this data could be used to make smarter decisions to improve patients' quality of life and to aid providers in making decisions about patient ongoing care. There are already technological tools for tracking and visualizing health data such as sleep (e.g., ), physical activity (e.g., ), variations in weight (e.g., ), and blood sugar level (e.g., ). However, most of these tracking tools are not designed to fully meet patients and healthcare providers' expectations and do not support reviewing patient-generated data with healthcare providers during clinical visits. One way to support patients in presenting their data with healthcare providers is to visualize the patient-generated data collections effectively. Yet, we lack an understanding of what type of visualization designs can support chronic patients to present and review their health data with healthcare providers during clinical visits. To answer this question, we explored patients' and healthcare providers' perspectives on presenting and reviewing patient data. To extract healthcare provider requirements when reviewing patientgenerated data during a clinical visit, we conducted a focus group with a mixed group of healthcare providers. To uncover patient stories and their approaches to tracking and presenting their health data, we interviewed eight patients with chronic conditions who actively track their health data. Our findings revealed four factors shaping patient-generated data: data items & data context collected by patients, time commitment invested by patients to track data, patients' motivation for collecting data, and patients' support circle. Considering these four factors, we designed various visualizations representing patient-generated data collections we gathered from our patients. Instead of pursuing a single generalized visualization design, we designed individually tailored visualizations for each patient. Based on our preliminary visualization designs, we proposed a design space of patient-generated data visualizations. Next, using these individually tailored visualization designs as elicitation artifacts, we interviewed the healthcare providers who had initiated the request for this project to reflect on the designs. Healthcare providers pointed to four use cases that they envision these visualizations could support their practice. As a whole, the of all our studies led to one message: the importance of designing patient-generated data visualizations by considering each patient and healthcare provider rather than designing for generalization. However, it may seem impossible to either design a unique set of visualizations for each patient or expect patients to design their own visualizations. We, as healthcare technology designers, need to provide patients and providers with a set of visualization designs as starting points. This approach would let each patient and provider choose the visualization designs that work the best for them with the capacity to customize the designs based on their lifestyle, conditions, collected data, and patientprovider relationships. Our contributions in this paper are as follow: We identified four factors shaping patient-generated data. We presented a design space of visualizations representing patient-generated data collections. We provided guidelines for designing future patient-generated data visualizations. In the this section, first we discuss patients' perspectives and goals for collecting their health data. In the second part, we provide an overview of healthcare providers' perspectives on the benefits and the challenges of using patient-generated data in their practice. In the last part, we discuss how technological and visualization tools can support patients and healthcare providers with presenting and reviewing patient-generated data collections. The number of patients with chronic conditions is increasing every day in the world. The nature of chronic conditions requires close monitoring and self-managing care for these patients. A survey study in 2013 showed at least seven out of ten adults in the U.S. track a health indicator for themselves or for someone whom they take care. An increase in the availability of wearable sensors, mobile health apps, and novel portable technologies provided patients with an extra boost to track more personal health data. People track their health data in various forms, including memorization, original artifacts, personal paper records, personal electronic records, and electronic patient portals. Patients track different types and amount of data depending on their personal health goals. These goals can range from preventing more complications, having more control over their health, setting personal health goals, improving their conditions, and sharing these self-collected data with their healthcare providers. Studies have shown that sharing patient-generated data with healthcare providers can improve patient-provider communication. Sharing health data also empowers patients in taking control of the conversation during a clinical visit and helps healthcare providers build a relationship with patients. Many patients share their self-collected health data with their healthcare providers during clinical visits seeking tailored medical advice. Some healthcare providers see value in patients collecting their health data and presenting them during clinical visits. They think that by reviewing patient-generated data, they will gain more insights into patient goals and will be able to provide more tailored care to patients. Providers think, in some cases, patientgenerated data might be more reliable than clinic measurements because the data is collected at more frequent intervals, and there is less recall bias. Providers mentioned that often, a hospital's electronic medical record system have misinformation or inaccuracies. In addition, patient data measured in the clinic (such as blood pressure) may be affected by the white coat effect and stress of the clinical environment. In these situations, patient-generated data can be used to reconcile these inaccuracies as patientgenerated data may contain less false-positive data than patient health data collected in the clinic. We should note that although healthcare providers may find patient-generated data complementary to clinical measurements and history taking if tracked in a meaningful way, they do not consider this data as a replacement to clinically measured data. Patients may not be willing to record their data when they have abnormal readings due to fear of consequences and may be worried that their data will be part of their permanent clinical record. In addition, providers sometimes express frustrations when patients do not track enough data, track excessive data, or track non-meaningful data. Patients also use different mediums and organization formats that work best for them to collect and present their health data. As a , the patient-generated data collections become heavily personal and complex, making it challenging for healthcare providers to understand and analyze. It is difficult to find the time to examine unrequested data during a short clinical visit. Most clinical visits are currently short. The common clinical visits with family physicians usually last about 10 to 20 minutes, leaving a short amount of time for reviewing patient-generated data. The providers may not find as much value reviewing patient-generated data during a clinical visit. Storing this data safely and securely can be challenging for providers and can add to their workload. Thus, there is still not a fully clear understanding of how, when, and what type of patient-generated data is most useful to review and discuss during clinical visits. One way to facilitate reviewing patient-generated data would be to have standardized data collection and presentation processes. However, a standardized process is probably not a panacea, as every patient and healthcare provider may have individualized preferences and needs. There is evidence that technology can support providers and patients in improving the quality of communicating patient data. Previous work raised questions about how technology should be designed that could assist both patients and healthcare providers in smoothly reviewing patient-generated data during clinical visits. One way could be visualizing these patient-generated data. Visualizing this data can benefit both patients and providers, if carefully designed so that it seamlessly integrates both perspectives into patient care planning. However, designing a general solution that works for all patients and providers is not easily achievable. Thus, first we need to move towards designing tailored visualizations, making an individualized visualization experience for each patient and provider. We were approached by a group of healthcare providers from a local hospital who are involved in the care of chronic patients to explore if, and how, to design technology that can enhance the process of presenting and reviewing patient-generated data during a clinical visit. To answer this question, we took an iterative design approach with close involvement of both patients and healthcare providers. The Institutional Review Board of (anonymized) University approved this study. First, we conducted a focus group with the healthcare provider that voiced concerns for reviewing patient-generated data. Then, to complete the healthcare providers' perspectives reviewing patientgenerated data,, we interviewed eight patients actively collect health data and collected a sample of their data. We asked our patient participants about their experience collecting, analyzing, and sharing their data with healthcare providers. Next, we leveraged this understanding to propose potential visualization designs representing patient-generated data that we collected. Our goal was to design visualizations to improve the process of reviewing these data during clinical visits. Last, we interviewed healthcare providers seeking their reflection on our proposed visualization designs. We also asked the providers how they envision using these visualizations in their practice. To clarify, confirm, and gain a deeper understanding of the healthcare providers' perspectives about the patient-generated data collection review process, we conducted a formal focus group with a mixed group of healthcare providers. Our focus group included a subgroup of providers who initially approached us including a clinical endocrinologist (with 21 years of experience), one internal medicine specialist physician (with 29 years of experience), and one healthcare provider (with 9 years of experience) directly supporting patients who monitor their data. Three other healthcare researchers were present during the focus group listening to the discussions. In our focus group, we asked healthcare providers about their experiences reviewing the patient-generated data, analyzing and understanding the patient data, and giving advice to patients based on their data. One interviewer primarily posed the questions during the discussion and two other researchers from our interview team took field notes. The focus group lasted around 60 minutes. We video-recorded, transcribed the focus group discussions, and later we used the grounded theory to analysis the data. To understand patients' perspectives on tracking and presenting their self-generated health data, we interviewed eight patients who suffer from one or multiple chronic conditions. We used several methods of recruitment for this study: emails from a local Patient Care Networks directors, Patient Care Networks newsletter ads, targeted recruitment through the healthcare provider who participated in the focus group and snowball sampling. We conducted an hour long semi-structured interview with each patient. We formed our patient interview questions based on the of our discussions during the focus group with healthcare providers. We asked participants to bring a sample of their data to the interview session and walk us through their data sample in detail. We video-recorded and transcribed all the interviews. To analyze the interview , we used the grounded theory method, analyzing each interview in a separate process. Our goal was to reach a deeper understanding of each patient's story. We state proof of existence for each interview and do not try to generalize our findings across patients. Next, based on the requirements of each individual patient, we sketched various visualization alternatives representing their own patient-generated data collections. As a group, we discussed the visualizations and how they meet the patients' needs. Then, we selected one or several alternative designs that best matched the patient's requirements. To complete our design cycle, we took our visualization designs back to three healthcare providers, who were among the group that initiated this project, seeking their feedback. We interviewed an internal medicine physician with 29 years of experience (C1), a clinical endocrinologist with 21 years of experience (C2), and a complex chronic specialist physician with 22 years of experience (C3). Each session lasted between 40-60 minutes and was video recorded and later transcribed. In the interview session, we first gave the providers a description of the patients' conditions, their personal stories, and their data collection processes. Then, we shared the visualization designs with the providers and observed their reactions walking through and talking out loud about the designs. From our analysis of the focus group transcripts, we extracted four requirements by our healthcare provider participants, to support reviewing patient-generated data during clinical visits. R1-Integrating data context: Healthcare providers think patient sometimes collect too many data items, but data without context is not helpful for medical purposes, "you get the data in a 7 by 6 table with numbers and they are all over the place. Without food information, stress information, activity information it does look like a bunch of noise. You don't see a pattern without being able to query on those other dimensions. Like your sugar is high, are you stressed?" (C1). To overcome this challenge, providers need tools that are able to integrate context with data. R2-Summarizing for quick presentation of data: Patients sometimes come to clinical visits with a large collection of data and expect their healthcare providers to help them make sense of their data "they clearly put in a lot of work, but you don't have time and you have nowhere to begin" (C1). Healthcare providers want tools with abilities to summarize and filter patient data to see trends, patterns, and anomalies. R3-Sharing goals and motivations: Our healthcare providers told us patients usually have different goals than providers which may cause conflicts. Patients often like to discuss details of their data, but providers are more interested in an overview of the whole data, so they wanted "a platform that forces people to be explicit between stakeholders" (C2). With this in mind, providers wanted to have tools with ability to overview and focus on parts of the data to explore the patient data in both focused and detailed views accommodating their goals and patients' goals. R4-Supporting conversations: Both patients and healthcare providers need support to discuss their concerns "[patient says] I have questions about [this] and the doctor says ok, great, that is what is going on there. But I am more concerned about this" (C1). Healthcare providers told us they need support opening up communications with patients which may have not happened otherwise; tools that can represent patient data in different views letting patients and providers discuss various aspects of patient data. The findings from the focus group helped us form our patient interview questions. Our healthcare providers found patient-generated data useful when patients collect meaningful data with context. Thus, in our patient interviews, we asked our patient participants to talk about the data items and the context data they collect. Our providers expressed their concerns about patients committing an excessive amount of time on data collection ing in large datasets. Thus, to get patient perspectives in this manner, we asked our patient participants to tell us about their time commitment to data collection. Our healthcare providers talked about the impact of patient goals and motivation on their data collection and data sharing. Thus, in our patient interviews, we asked patients to tell us about their goals and motivation for collecting data and if they were advised to track data by their providers. Our healthcare providers saw value in having a patient's presence during clinical conversations. Thus, we asked the patients whether they shared their data with their healthcare providers or their caregivers at home and how was their experience getting support. To design the patient-generated visualization designs, we considered the four requirements (R1-R4) identified from our focus group and followed the design guidelines established in the literature. To accommodate data context integration, R1, in the visualization, we used "Tooltip" which is an identifying tool presenting the attribute data attached to an object. To incorporate R2, we followed the basic information seeking principles. To fulfill R3, we incorporated "overview and details-on-demand" interactions in our designs. To support patients and providers view patient data from different perspectives, R4, we designed multiple visualization designs for each patient-generated data collection. We allocated pseudonyms to confer the anonymity of our patients. In each part of this section, we first present the profile of the patient; their data and context, their time commitment, their motivation, and their support circle. For each patient, we ideated and designed one or multiple visualizations. These visualization designs are simple visualizations that are carefully designed to capture providers' requirements and each individual patient needs and may not be novel designs by themselves. We explain the detail of each visualization design we sketched to display patient data and how we took the providers' and patients' requirements into considerations when exploring visualization design opportunities to represent their patient-generated data collections. We did not restrict ourself to designing a certain number of visualization representations; we sketched as many design possibilities as we could think of to present the data for the patient. In total, we generated 20 preliminary visualization designs for eight patients. We laid out these designs on a design space board (Fig. 1). In this design space, each column corresponds to one patient and the visualizations in the column are design variations for that patient. Later, as a group, we discussed all of the visualization designs and selected the designs that best represent each individual patient. In this figure, the selected designs are highlighted with an orange border. We acknowledge that these designs are not the only possible visualizations and other designers/researchers may come up with variations to these designs. Here, we present our designs and we hope this will be a starting point for other researchers and designers to contribute more patient stories to the literature and to move towards thinking about designing more for individuals. Maria is 67 years old, one day she experienced high blood pressure and visited the hospital emergency room. After that hospital visit, Maria constantly experienced high blood pressure. That year, she was diagnosed with hypertension. Data & context: Maria was advised to track her blood pressure and heart rate on a regular basis using a cuff machine. She uses a notebook to record her readings (Fig. 3 -a). We designed a visualization representing both Maria's blood pressure and heart rate readings (Fig. 1 -column p#1 -first row). We display blood pressure readings in the form of bars and show the patient's heart rate on demand. Each bar represents one blood pressure reading, we associate the bottom border of the bar to diastolic and the top border of the bar to systolic. The two horizontal lines in the show the normal blood pressure reading range (120 over 80). In addition, we added colour to each bar showing a normal (green), an abnormal (yellow), or a dangerous (red) blood pressure reading. Time commitment: Maria tracks her blood pressure and heart rate three to four times per day. Thus in our design each bar in the visualization shows one reading with the time of the recording. Motivation: Maria's ultimate goal for tracking her data is "to feel better... make my blood pressure go down" (P01). After her diagnosis, she changed her life style to reach her goals. She is drinking more fluids and reduced the amount of salt in her diet. She is hopeful that she can reach her goal. She also keeps a record of events or activities she thinks may be relevant to her blood pressure, so later during a medical visit, she can discuss them with her healthcare providers. Thus, in our design we have an option to add notes associated with her blood pressure records. Support circle: Maria presents her notebook to her family physician saying, "because of this [notebook], it will be easier for me to inform the doctor" (P01). She hopes her family physician can make sense of the data and make adjustments to her treatment plans based on her data. To accommodate Maria's need for sharing her data, we designed this visualization with the capacity to show an overview of blood pressure readings over months (top row in the design) to quickly check her overall status in the past months as well as detailed numbers on demands (bottom row in the design). Andrew was diagnosed with type 1 diabetes about 16 years ago at the age of 52. Due to his age, he was first misdiagnosed with type 2 diabetes. After his diagnosis, his interaction with the healthcare system changed from visiting his family physicians once a year to getting an A1C test every three months. He has been in direct interaction with a nurse educator, a foot care clinic, and an endocrinologist in a diabetic neuropathology clinic. Data & context: Andrew measures his blood glucose and basal rate as advised by his nurse educator and endocrinologist (Fig. 3 -b). He uses a glucose meter to measure the concentration of glucose in his blood and an insulin pump to calculate the amount of insulin required. We represent Andrew's blood glucose data in two different visualization designs. The (Fig. 1 -column p#2 -first row) is a detailed view of one day of Andrew's glucose level. The circle shows a 24-hour clock. Each time Andrew measures his glucose, we show his reading on that time on the clock with a bar. The height of the bar represents the glucose rate and the color of the bar represents the normality of the glucose rate; if the glucose reading is too low (red), low (yellow), normal (green), high (yellow), or too high (red). In the (Fig. 1 -column p#2 -second row), the top part shows all blood glucose ratings recorded in a month with circular points. The y-axis shows the glucose rate and the x-axis shows the date. We also double coded each data point with the same colour themes as the first design. Time commitment: Before each meal, Andrew measures his blood glucose using the glucose meter and enters his readings into the insulin pump. The pump automatically send Andrew's insulin intake to his nurse educator. Besides that, Andrew keeps track of his basal rates that he measures using the glucose meter, in a notebook to later share with his nurse educator. Every time Andrew visits his healthcare providers to check on his conditions, he shares the recorded data he collected over the past few months with his healthcare providers. Thus, in our visualization designs, we included a weekly or monthly overview of his glucose rates at the bottom of both designs. Motivation: Andrew lives a good life, eats healthy, gets enough sleep, and has a balanced work-life lifestyle. He recently got diabetes complications. After experiencing the complications, he is hoping to start an exercise routine. Andrew tracks his exercise on the side to understand the effect of his physical activities on his blood glucose. Thus, we added an option for the patient to add a free style note (e.g., exercise) on his data point to appear on demand when hovering over the data point in the visualizations. Support circle: Andrew has a hard time analyzing and finding trends in his data to adjust his lifestyle saying, " There's so many factors that come to play with your blood sugars and trying to get everything in the right spot" (P02). He expects his healthcare providers to make sense of his data for him and give him direct instructions on how to better manage his conditions. Thus, we included a weekly and a monthly view of the glucose recordings on the bottom of both designs to give an overview of his data. Jen is 34 years old and was diagnosed with hypertension when she was 18 years old and was medicated for a few months. Last year, she had a visit with her family physician to get treatment for an infection and her blood pressure reading was high at the clinic. But, when she checked her blood pressure at home, she noticed that her reading was closer to normal readings than in the clinic. Data & context: Jen tracks her blood pressure and heart rate (Fig. 3 -c). Since she is experiencing a steady heart rate, she mainly focuses on her blood pressure data. Thus, we only display her blood pressure data, in two different visualization alternatives. We designed two visualizations displaying Jen's blood pressure data. In the (Fig. 1 -column p#3 -first row) design, we have designed a tree based visualization with the ability to expand on demand. The top root represent the average blood pressure readings of the patient over one year. The next level shows the seasons, then months, and lastly the daily blood pressure reading. Jen uses three different colors to distinguish her readings into normal, borderline, and abnormal. With colour coding her numbers, she can quickly glance over her data. We have used the same idea in our visualization design and color coded her blood pressure readings. In the (Fig. 1 -column p#3 -second row) design, each bar shows an average of all Jen's blood pressure readings in a day, where the colours indicate the normality of the number. Dark green indicates high blood pressure readings, green indicates a normal blood pressure readings, and light green indicates low blood pressure readings. Looking at this view, she can decide if she is having more dark or light colors in a period of time. Whenever she decides to focus on a certain period of time, she can select that section and a table view appears underneath with data displayed for each day. Time commitment: Jen has been measuring her blood pressure a few times per week for a year and believes her condition is under control with steady normal blood pressure readings: "Lately, it's been quite good for the last several months. So, kind of since January I check it maybe once a week now as opposed to every day"(P03). Thus, in our designs we only display maximum one reading per day. Motivation: Since her last clinical visit, Jen monitors her numbers to prevent any complications or developing hypertension for the second time. Last time she was taking medications for her hypertension, she experienced many side effects, and she fears that the healthcare providers may medicate her again: "I've been borderline and they've talked about medicating me for it, but I would rather not be if I can avoid it. So, I am just trying to manage it other ways before getting to that point" (P03). Support circle: Jen usually does light exercises, gardening, or short walks to stay healthy. To stay under 1500 mg sodium per day she plans her weekly meals with her husband. She expressed her concerns to her physicians that she only has high blood pressure when she is at the clinic, visiting her providers gets her anxious and stressed. To overcome this problem, she writes notes next to her readings keeping track of any triggering factors such as a clinical visit. She is hoping by showing the numbers she tracked at home to her healthcare providers, she can tell them, "No, it's usually right around 120/80. It's not always this high" (P03). Therefore, in our designs, we have an option for Jen to mark the blood pressure readings measured during her clinical visits. Lucas is 43 years old and suffers from hypertension, type 2 diabetes, and depression. Lucas was hospitalized a few times with suicidal thoughts and high blood glucose. Tracking his blood pressure and glucose level helps him get his conditions under control; however, sometimes he experiences an emotional break down when his readings are higher than the normal range advised by his providers. Data & context: Lucas collects his glucose, blood pressure, and heart rate (Fig. 3 -d) in his notebooks. We designed two visualizations displaying all three items he is tracking (Fig. 1 -column P#4). Lucas wants to look at his glucose, blood pressure, and heart rate data all at once. Thus, we display all his data in one view. Each data point is color coded in both visualizations based on the ranges defined for Lucas's conditions. Green indicates normal, yellow shows borderline, and the out of range readings are colored in red. Time commitment: Lucas was advised by his providers to record his data five times a day. However, he is dealing with a lot of pressure due to his conditions and his personal problems, so he only manages to track his data once a day. Thus, in the (Fig. 1 column P#4 -first row), each vertical division in the chart shows one data item: blood pressure, blood glucose, and heart rate. In the (Fig. 1 -column P#4 -second row), we show each day of data readings in a flower shape visualization, each petal representing one data item: blood pressure, blood glucose, and heart rate. Motivation: He feels frustrated and upset with himself for not having his conditions under control. Lucas hopes to get support that motivates him to track his data, but does not want to be pushed. He wants to exercise regularly, as it can help him stabilize his blood pressure and glucose level; however, his busy schedule does not allow for exercise. Instead, he tries to go for short walks to lower his blood pressure when he experiences high blood pressure. His goal is to get off the insulin by next year. Support circle: Lucas feels that he does not have enough family support and his family lacks compassion and doesn't understand the seriousness of his conditions. He has difficulty making sense of his data and expects his healthcare providers to understand his data and give him advice based on them. For instance, he was hoping to find relations between his blood pressure readings and glucose level, but could not find any correlations. Thus, we visualized all the three data items he collects adjacent to each other in one view. Ken is 37 years old and suffers from multiple conditions. He had memory problems, paranoia, and learning difficulties since childhood. He was diagnosed with behavioural disorder in 2005, mental health problems in 2009, and asperger syndrome in 2011. In addition, Ken has digestive problems and is experiencing pain in different parts of his body (e.g., neck, back, shoulder, ankle), which have not been officially diagnosed. Data & context: Ken tracks his nutrition data and symptoms related to his stomach pain and bowel movements using MySymptoms app. He tracks his pain to help with diagnosing the source of his pain (Fig. 3 -e). To understand the effects of his mental state on his conditions, he also tracks his mood. Ken prefers using multiple apps on his tablet to record different health data items, therefore we also visualized his data in two separate designs. He is happy with the app he uses for tracking his nutrition, so we focused our designs on the other data items (moon and pain). We sketched a visualization displaying Ken's mood data (Fig. 1 -column P#5 -first row). Each day on the calendar shows Ken's mood of the day which is colour coded; happy (green), normal (yellow), sad (red), and selfdefined (blue). We sketched a second visualization representing Ken's pain data (Fig. 1 -column P#5 -second row). Time commitment: Ken tracks his mood once everyday. To present his mood data, in the calendar visualization, we also allow for one mood entry per day (Fig. 1 -column P#5 -first row). On the other hand, the pain body mock-up visualization (Fig. 1 column P#5 -second row) does not display any time data and lets Ken record as many occurrences as he experiences. Each ring in this visualization represents pain experienced once in the marked location of body. Motivation: Ken's goals are to eat healthier, get more physically active, and lose weight. Also, he is hoping to get more involved in his care. He records relevant context as notes that he thinks may trigger his mood. Thus, we added an option for him to add free style notes to keep track of the context associated with each day in the calendar view visualization. Support circle: Ken tracks several symptoms and trigger factors that he thinks may be helpful for improving his health, but his healthcare providers do not always find his collected data useful. He is confused about which data items are useful to collect: "I gave all my symptoms to her, all recorded on a sheet. She said,'Oh, we're just looking at the gut issues.' I'm like, What about the rest?" (P05). Sarah is 49 years old and was diagnosed with type 1 diabetes in 1984. In 2013, Sarah was hospitalized experiencing severe gastroparesis symptoms. Later, Sarah developed arthritis in her hand and gets cortisone shots, which increases her glucose level after each shot. Data & context: She uses an insulin pump to manage her diabetes (Fig. 3 -f). Thus, in our visualizations, we represent her blood glucose data (Fig. 1 -column P#6 -first row). Since Sarah has an insulin pump, the device automatically tracks her blood glucose many times in a day. Thus, to visually show all the data points measured by her insulin pump in a day, we designed a clock visualization. The clock view can show all the data readings in one view with their timestamp. Sarah's healthcare providers predefined a normal range of glucose level for her based on her conditions. In the (Fig. 1 -column P#6 -first row) visualization, the blood glucose reading is marked with an X inside each ring and colour coded to green, yellow, and red based on the ranges defined for her. Time commitment: The pump automatically tracks her blood glucose level in different time intervals to program her insulin. Sarah does not regularly record her food intake, but when she feels sick, she takes notes in her phone of what she ate and her activities that may have affected her glucose: "there's really no answer, I've been dealing with this for about two or three years now " (P06). Motivation: Sarah has changed her lifestyle especially after her diagnosis with gastroparesis. She takes an active role in managing her conditions. She says "with gastroparesis there's no medication, there's no cure... it's a matter of just doing a lot of research and reading in different avenues (P06). Sarah has a fear of getting sick to the extent that she needs hospitalization. Support circle: Her diabetes nurse monitors Sarah's glucose level regularly. On the occasion that Sarah feels sick or in need of help, she calls her nurse and asks her nurse to log into her pump remotely. Based on her pump , the nurse will give her advice on how to normalize her glucose level. To discuss her readings over a week with her nurse, we displayed an overview in form of seven rings (days) (Fig. 1 -column P#6 -first row). Tim is 56 years old and was diagnosed with type 2 diabetes about 8-10 years ago. His condition has gotten worse in the past two years. Tim has been also dealing with hypertension for a long time. He also has a genetic disorder, Hereditary Hemorrhagic Telangiectasia that cause abnormality in blood vessel formation, but it does not affect his chronic conditions. Data & context: Tim uses a glucose meter to measure his glucose reading and records his readings in an app on his phone (Fig. 3 -g). He also uses a blood pressure cuff machine to measure his blood pressure. He then manually enters his blood pressure readings into two different apps on his phone since he is afraid one app will wipe out his recorded data. He prefers to collect his data on his phone rather than the booklet he was given by the nurse. We display both data items in one design (Fig. 1 -column P#7). Tim takes notes keeping track of events and special occasions (i.e., holidays and parties). Thus, to accommodate recording these notes, we added an option in our design to track and later display the notes. Time commitment: He tracks his blood glucose once or twice a day and measures his blood pressure a few times a day at different times. Thus, we also show multiple data readings on the chart per day. Tim normally skips tracking his data during vacation times. However, not tracking his data during his last vacation caused an abnormality in his data: "I was good for a while. Then took a vacation and, whoaa!"(P07). To visually display the effect of not tracking data we show the missing dates with dashed lines. Motivation: After visiting a new physician, the physician changed Tim's hypertension medication to a more recently developed medication. Since the change of his medication, his blood pressure has been generally stable and he got motivated to start tracking it, "I kicked myself, I should have tracked it longer"(P07). He is hoping to become more active in his care. Tim has a standing order from his diabetes nurse to get A1C test every three months. He is hoping his glucose level goes below 6.5: "six months ago, it was 8.1. Now it's 7.1"(P07). To make it easier for him to check if his numbers are normal, we colour coded (green, yellow, red) the data points. Support circle: Tim's diabetes nurse and his family physician automatically receive the of his A1C test. However, Tim does not share any of his self-collected data with his providers. Katy is 52 years old and she suffers from hypertension, asthma, arthritis, chronic pain, and depression. She was diagnosed with asthma 21 years ago which is mostly under control with medications. In 2004, she gave birth to a premature baby and had a sudden death in her family. Later that year, she was diagnosed with severe depression and was hospitalized in the psychiatric ward. Data & context: As a of Katy's depression, she gained 150 pounds. Three years ago, she joined a weight management group and was advised by her dietitian to track her food intake (Fig. 3 -h), but Katy does not like to share her collected data with her providers. A few years ago, Katy started to experience pain in certain areas of her upper body; however, her physician did not believe her pain was real and was dismissive to her condition. After struggling with pain for a while, she decided to look for another pain specialist. She created an Excel sheet with upper body part names and each day she would put in a number corresponding to her pain level in addition to the type of pain (stabbing, stinging, and shooting). Thus, we designed an upper body mock-up drawing visualization to help her visually track the type and location of her pain (Fig. 1 -column P#8 -second row). In the visualization, the tensity of the pain is represented by the number of rings (1 to 10) and the three types of pain that Katy tracks are distinguished with different colours. Time commitment: Every time Katy experiences pain, she records her pain data. Thus, we also allow for as many (pain) data entry as pain occur during a day in our visualization design. Motivation: Katy writes side notes to her pain data to investigate if there is any relationship between the time of the day, her activities, and her pain level. She shared her pain diary with her new pain specialist to see if there is any relationship between the time of the day, her activities, and her pain level; Katy told us her specialist said: "This is great, there is no relationship to anything which just tells me it is probably a nerve or something. This is fabulous and I want to keep this!" (P08). Support circle: Katy hoped to receive more tailored care by sharing her self-collected health data with her healthcare providers. She sees value in tracking her health data and sharing them with her healthcare providers. We display an overview to her pain data by displaying a week of her pain data in form of small body mockups at the bottom of our design. This view will help providers to get an overview and to find possible patterns or trigger factors. We presented the patient-generated data visualizations to three of the providers who initially requested visualizations and technological support for reviewing patient-generated data. We observed providers' reactions towards our visualization designs and asked for their feedback. The providers varied widely in why, when, and how they want to use patient-generated data visualizations in their practices. We present our according to the two themes we identified through analyzing the interview data. The themes are 1) the visualizations' use cases in providers' practice, and 2) the platforms for implementing the visualizations. Providers envisioned different use cases for the visualizations in their practice: 1) one provider saw value in the use of these visualizations only by patients, 2) two provider wanted to use them to review patient data during clinical visits collaboratively, 3) one provider thought of using them to support their medical judgment, and 4) two providers found displaying patient data through different lenses useful to understand the data better. Encourage patient self-experimentation and goal setting: The complex chronic care specialist, C3, expected visualization views that would encourage patients to do more self-experiments. He thinks particularly for chronic symptom management where there is no complete treatment to resolve the symptoms, but rather it is a matter of trying to track and manage them, experimenting to find trigger factors can be helpful for these patients. The potential of self-experimenting with data can help patients find solutions to perform easier through their everyday life. In addition, C3 thinks visualization designs need to have the capacity to support patients in setting goals and tracking an intervention that patients may set in their minds to control their symptoms.: " For example, taking three glasses of water per day may reduce headache" (C3). Although this provider was interested in encouraging patients to do selfexperiments and set goals, C3 wanted patients to share the data collections with them. In these circumstances, the providers can help patients understand if there is a scientific correlation between variables and help patients understand the body mechanism that might explain this correlation. Juxtapose data for collaborative interpretation: The internal medicine specialist, C1, was cautious about juxtaposing all patient-generated data items in a single visualization view. He was concerned that juxtaposing patient data could imply a link that may not exist and falsely medicalize the relation between the data: " the minute you put them on a shared data exhibit, it is a correlation" (C1). Although he was not enthusiastic about presenting some data items such as blood pressure and glucose level in one view, he found coupling some data points useful. For instance, when seeing the (Fig. 1 -column p#6 -first row) visualization he was keen to view patient food intake and blood sugars displayed together to investigate their relationship. Another functionality that the providers found useful was the potential to overlay data collected across different situations or days. By overlapping patient data, providers may be able to find patterns in patients' data. For instance, C3 was interested in overlapping patients' glucose data over a few days to find out the effect of biking for 30 minutes on patient glucose levels, "the nature of the adjustments is very rarely a single day" (C3). Offer a holistic overview to the provider: The endocrinologist clinician, C2, showed interest in a holistic visualization view of all the data items a patient collects. She found the visualization designs that represented all patient data items in one view very useful for planning complex chronic patient care. For example, for displaying blood pressure and glucose level in one view she said: " as a care provider, I can show that'yeah, during these times these situations are really bad for you' " (C2). She was also keen to see patient's physical activities such as steps taken per day presented in the same view to understand the effect of exercise on the patient's other health conditions. C2 was interested in having access to the patients' notes describing the context and the situation when this data was recorded. She told us that she encourages her patients to take notes of their emotional states, their meals, or any other relevant information when recording their health data. Knowing the context associated with the data, the provider has more information to make informed medical judgments. Understand data better through a different lense: Providers were able to quickly adapt to new visualization designs and warmed up to the idea of alternative views of data, promising for adoption in their practice. For example, C3, said he appreciates the visualizations capability to display patient data differently. He said both patients and providers are used to seeing patient data in a standard tabular format. He thought showing patient data in different forms will give patients extra support in understanding their data and taking actions towards enhancing their health: " We never had this kind of things [visualizations] and so, this is where the notion of'same data, different lens' becomes useful" (C3). The providers also recognized that some of these visualization designs can be used to represent other health measurements. One of the providers who was at first skeptical of using the blood pressure tree design (Fig. 1 -column p#3 -first row), after reviewing and discussing the design, suggested using this visualization for collectively displaying 24-hour blood pressure cuff machine data. Providers use these machines to closely monitor the patient blood pressure to help with diagnosis: "this is an attractive idea, maybe this kind of visualization can be used for a 24 hour report [showing data] every 10 minutes" (C1). The choice of visualization platform can make a difference in designing the right patient-generated data visualization. The providers talked to us about their preferred patient-generated data platforms, and the rationales, benefits, and trade-offs of their choices. Different technology and platforms for implementing such visualizations include data booklets, websites, phone apps, and patient portals. Booklets: To smoothly integrate visualizations into providers' practices, one challenge is to design the patient-generated data visualizations compatible and aligned with the current providers' practices. Providers usually give patients tabular template booklets to record data. C3 mentioned that he preferred reading patient data in these booklet format, since it is easier and faster for him to find trends. Printed visualizations in the form of booklets can be familiar and easy to use for providers, but do not support interactivity. Websites: Some providers prefer to have patient data uploaded on designated websites where providers could potentially integrate patient-generated data into the patient's health records. C2 thought that, if designed well, a website would be a good platform that could support both patients and providers to interact with patient-generated data and see the data in different ways. However, healthcare services usually have restrictive policies for use of websites in clinical settings. Phone Apps: Patients may not feel comfortable sharing all the data with one healthcare provider and may only be willing to share related data with a specific provider depending on their specialty. C2 thought that using a personal phone to record data could be a solution, since patients have full authority to share data they wish. However, small display real estate could cause limitations in designing visualizations that represent all patient-generated data at once. Also, sharing a small display between patients and providers during clinical visits can be difficult. Patient Portals: Providers normally have a PC in their clinical rooms for taking specific notes about a patient's condition and recording them in a patient's healthcare portal. C1 was keen on the idea of asking patients to link their self-collected data into their healthcare portals ahead of time. He thought that having patientgenerated data collections and visualizations available on the portal could not only save time, but could also be easily accessible for discussion. However, implementing visualizations into these portals can be a long and difficult process, which also requires support from healthcare services. Effective communication of patient-generated data during clinical visits can help patients feel understood and support healthcare providers get all the necessary data they need to make proper medical decisions. Our objective was to design visualizations to support patients present patient-generated data for reviewing during clinical visits. The focus of our studies was on studying patients with chronic conditions and the healthcare providers who visit chronic patients. The of our patient interview studies revealed the individualities and the complexities of patient-generated data collections. Each patient has a unique body, a highly individualized lifestyle, a different set of goals, and a personalized patient-provider relationship. All these factors need to be considered while caring or designing for patients. How can we design only one visualization solution that can consider all these differences in patients? Providers also differed in their principle goal of using patientgenerated data. This has major implications on the design of visualization. A solution that works for one provider may not work for another. This may affect the types of visualizations we consider for them and their patients. There are many driving forces for designing effective patient-generated data visualizations. It is still unclear which direction works best for both patients and providers. In software and technology design, research, and businesses, there is often the notion of designing with a generalization mindset,'one-size-fits-all'. The idea of designing one software or one visualization tool that can address everyone's problem may be appealing and cost-efficient, but it does not always bring validation. We echo the call from recent research for the necessity to design for particulars, individuals. Looking into medical literature and the approaches taken in healthcare services for patient care planning, we often see one-toone interactions between a patient and their healthcare providers in clinical visits. This one-to-one interaction model has been practiced for centuries in medicine and is tailored depending on the individualities of each patient and their healthcare provider. Similarly, for designing visualizations to improve patient-provider communication, we, as visualization and technology designers, should take directions from the medical literature and their practices. We should take steps towards designing individualized tailored visualizations based on both patient and provider preferences to be able to accommodate as many patient-provider communications as possible. Perhaps one solution can be to start designing and developing many patient-generated visualizations tailored based on both the healthcare provider and the patient preferences. There have been attempts in the literature to design visualizations representing patient-generated data for some chronic conditions including, visualizing bipolar patient lived experience, collaborative interactive storyboards design with chronic pediatric patients, photo-based visualization to support patients with irritable bowel syndrome communicate with providers. However, designing visualizations to support chronic patients with their self-collected data is indeterminant, or in other word a wicked problem, meaning there are no definitive solutions or limits to this design problem. Our design study in this paper is a step towards tackling this wicked problem. We followed two criteria required for a rigor design study when addressing a wicked problem: 1) A rigor solution to a wicked problem needs to include multiple perspectives to shape the design problem and consider a broad set of solutions. Thus, in our study, we included the perspectives of both patients and healthcare providers. We explored the healthcare provider perspectives on reviewing patient-generated data during clinical visits and the details of eight patients' approaches to tracking and presenting their health data. Furthermore, we looked into a sample of our patient participants data collections to understand patients' methods of recording their data and their reasoning. 2) A design study is not and should not be reproducible; rather the solutions proposed are one of many possible solutions. However, a rigor solution to a wicked problem needs to report the process of design and analysis in a transparent way. We designed multiple alternative visualizations for each patient. All of our visualizations together shaped a design space of variant patient-generated data representations. We understand that depending on the patient needs, the providers' expectations, and the patient-provider relationship dynamics, a different set of visualization designs could be suitable. Our solutions are one of the many possible solutions to this wicked problem. Furthermore, we explained the process of design and reflection of our design in detail transparently. We hope that the detailed design process we provided supports other researchers and designers to further tackle this wicked problem and to design patient-generated data visualizations. Inspired by the four factors identified from the focus group and the patient interviews, and the healthcare providers' reflections on the designs, we provide the following design guidelines. We understand that these guidelines are not exhaustive; rather these are starting points to design patient-generated visualizations. Include data context in the design: Patients often track a large amount of data. To gain valuable insights from this data, the patients often take notes of the events, circumstances, or emotions associated with the data points. On the other hand, healthcare providers in our study found this contextual information useful to make medical decisions. Previous studies also pointed to the importance of relevant dimensions and context of data for making medical . Thus, visualization designs need to allow for smooth inclusion of the contextual data often in forms of free-format text along with the data points. Consider patients' time commitments in the design: Chronic patients often deal with many issues in their everyday life, leaving them with less free time to track and record their data regularly. The Apps available on the market do not usually consider patient differences in the time they invest in collecting data. For instance, displaying empty entries can cause mental effects, making patients feel they are not doing enough. Thus, visualization designs should allow patients to customize the amount of information and input fields being shown. Allow patients to freely explore their data: Our show that the patient's motivation for tracking and presenting their data to providers play an important role in the design. Some patients are eager to find correlations between their data, some patients are looking for causation of their symptoms, and some patients want to have an overview of their numbers. Previous work also stressed the need to support patients in sense-making and problem-solving with data. Thus, these differences in patients' motivations for data collection should be considered when designing visualizations to represent patient-generated data. Support patients' needs to (partially) share their data: Patients differ in the support they receive from their family, friends, and the healthcare team. Some patients benefit from sharing their whole data with their support circle, some are interested in sharing a selection of their data, and some are hesitant to share their data. Thus, visualization designs should support sharing overviews, selective views, and protected views. Visualization designs that support sharing views need to also include annotation capability and multiple views (e.g., patient view and clinician view). Support providers interacting with patient data: Although providers had different perspectives on the use cases of patientgenerated data visualizations in their practice, they had commonalities in regards to necessary interactive functionalities. All of our providers talked about how difficult it can be to cope with messy, inconsistent, and complicated data collections. This suggests that ata-glance data comprehension is an important visualization design goal. In addition, providers needed interactions to better understand the data including filtering the data, focusing into data details, and overlaying different parts of the data for comparison. In recent years, we have seen growing interests among patients with chronic conditions to track and analyze their data. However, sharing this data with healthcare providers can be challenging due to limited time in clinical visits and the large and complex nature of patient-generated data. We responded to a group of healthcare providers' call from a local hospital to design potential technological solutions to address the challenges of presenting, reviewing, and analyzing patient-generated data collections. We first gained healthcare providers' perspectives through a focus group. Then, we took an in-depth look at chronically ill patients' perspectives tracking their health data. The individual differences among these patients promoted a design space approach where we used insights from these patients to design a space of possible tailored visualizations. By exploring the possibilities of designing individual tailored visualizations representing patient-generated data, we have added one way that can support patients and healthcare providers when reviewing patient-generated data during clinical visits. We hope our proposed visualizations provide patients and healthcare providers better opportunities to present, review, and gain insights on patientgenerated data. We note that we included the perspectives of a small number of patients and healthcare providers; thus, other perspectives may not be included in our . However, we envision this study as a stepping stone for the call to focus more on designing technologies in healthcare for individuals. We encourage the human-computer interaction, visualization, and healthcare communities to repeat these studies by including more patients and healthcare providers and explore designing tailored visualizations for each individual. Then, as a community, we can move towards accumulating these perspectives and designs to empower individuals with accessible design variations. We hope that in the long term, the of this exploration contribute to supporting patients' and healthcare providers' in reviewing patient-generated data collections using visualizations during clinical visits.

represent this text to find a much much longer article about it with say hundreds-thousands of words

We explored the visualization designs that can support chronic patients to present and review their health data with healthcare providers during clinical visits.

Represent the TLDR for finding the corresponding scientific article:

This paper explores using wearable sensory augmenting technology to facilitate first-hand perspective-taking of what it is like to have cat-like whiskers.


Question: Represent the article for finding a single-sentence TLDR

Question: Recently, neural-network based forward dynamics models have been proposed that attempt to learn the dynamics of physical systems in a deterministic way. While near-term motion can be predicted accurately, long-term predictions suffer from accumulating input and prediction errors which can lead to plausible but different trajectories that diverge from the ground truth. A system that predicts distributions of the future physical states for long time horizons based on its uncertainty is thus a promising solution. In this work, we introduce a novel robust Monte Carlo sampling based graph-convolutional dropout method that allows us to sample multiple plausible trajectories for an initial state given a neural-network based forward dynamics predictor. By introducing a new shape preservation loss and training our dynamics model recurrently, we stabilize long-term predictions. We show that our model’s long-term forward dynamics prediction errors on complicated physical interactions of rigid and deformable objects of various shapes are significantly lower than existing strong baselines. Lastly, we demonstrate how generating multiple trajectories with our Monte Carlo dropout method can be used to train model-free reinforcement learning agents faster and to better solutions on simple manipulation tasks. Small errors in the input and prediction can lead to significantly different object trajectories. The orange ball could either end up on the left or right side of the wedge. Learning to predict the physical motion of objects from data is an open area of research. Yet, recent (hierarchical) relation network based forward dynamics predictors (; ;) seem to be a promising alternative to conventional physics engines that are key components of robot control, computer vision and reinforcement learning (RL) systems. Physics simulators, both traditional numerical solvers and learned prediction models, still suffer from insufficient accuracy in challenging scenarios. Small errors in the input and model can lead to dramatically different object trajectories. Take the orange ball that is falling on the blue wedge in Figure 1. Depending on where the orange ball starts or what bias the model has, the ball could either end up on the left or right side. Both are valid outcomes. However, deterministic physics engines will either predict one trajectory or the other. While it is important to reduce errors in each prediction, it is also important to acknowledge that uncertain situations might not have one but multiple possible outcomes. In machine learning, uncertainty-aware neural networks avoid deterministic point estimates by predicting distributions or by randomly sampling in the prediction interval. In the context of dynamics predictions, we propose to use Monte Carlo sampling based dropout on the model weights of a learned forward dynamics predictor to model uncertainty and sample multiple plausible trajectories for an initial state. To stabilize each trajectory and reduce error accumulation over long-time horizons, we use a state-invariant recurrent training mechanism. By feeding back predictions as input over multiple time steps, the model becomes more robust to its own prediction errors without the need for a hidden state. Finally, we introduce a new shape loss on the model predictions that constrains the pairwise distances between objects and object parts and greatly improves shape preservation and the stability of trajectories over long-time horizons. Our final fully differentiable forward dynamics model is able to sample multiple, more accurate and more stable trajectories over long-time horizons compared to existing baselines. An accurate forward dynamics predictor that is able to predict a distribution of future states can be of great importance for robotic control. In model-free reinforcement learning, accomplishing tasks through random exploration is sample inefficient and hardly generalizable. Model-based methods promise greater generalization abilities, but suffer from deterministic world models that are hard to learn and fail in stochastic environments. With our stochastic forward dynamics predictor, we can move part of the sampling process into the environment, physically grounding the random exploration of model-free agents. As the agent is able to observe multiple trajectories at a given state without actually executing multiple actions, the sample efficiency is greatly improved while the stochasticity of each state and action is implicitly learned. We show on several control experiments that a model-free agent trained in our stochastic forward dynamics environment is not only able to better explore and learn faster but often also comes to better solutions than agents trained in deterministic environments. In summary, we propose a stochastic differentiable forward dynamics model that is able to generate multiple plausible trajectories via Monte Carlo (MC) based graph-convolutional dropout. We greatly improve the accuracy and stability of long-term predictions by proposing a new fullyconnected shape loss term and training the model recurrently end-to-end in a state-invariant way. We demonstrate how our stochastic dynamics model can be used to improve the efficiency and performance of model-free reinforcement learning agents on several physical manipulation tasks. Physical dynamics prediction has long been an open research questions (; ; ; ; a; ; a;). Recent advancements in deep learning allowed for emergence of successful systems that aim at solving this problem by learning from data. and proposed a graph-based approach with object-centric and relation-centric representations, and a neural network architecture that predicts object dynamics and interaction between objects in complex 2D scenes. implement a relational network with particle representation for objects, but extend this approach to 3D scenes introducing hierarchical graph representations for computational tractability. These works rely however on a single step prediction during training. We propose a recurrent training scheme on multiple step predictions and show lower long-term error in our experiments. Simulating future plausible object states under physical and user constraints is a commonly addressed challenge in computer graphics. analysed uncertainties in their simulation model for multi-body scenarios and used a Markov chain Monte Carlo algorithm to predict multiple trajectories. based their work on psychological findings about human errors in predicting object dynamics and simulated multiple future environment states by applying external random impulses to colliding bodies. Trajectories generated by both of the mentioned methods are visually plausible to human, but can often diverge from the real physical behavior and require extensive expertise to choose simulation parameters that ensure convergence. support the psychological theory behind the latter work, further suggesting that human predictions of non-linear dynamic effects such as collisions are far from perfect, and thus allow for a less advanced perturbation methods. draw a line between physical and visual plausibility naming further factors improving the visual plausibility of a scenario such as the number of simultaneous collisions or homogeneity of colliding objects. In physical systems, situations that are non-intuitive to human can occur due to an unobservable state of the environment, e.g an object colliding with a fast rotating wheel or an unexpected behavior of a compressed spring. This opposes the goal of computer graphics were visual plausibility becomes a stronger requirement and motivates the search for more sophisticated methods for sampling probable states in physics engines. Multiple techniques allow neural networks to incorporate uncertainty in model predictions. MeanVariance-Estimation is a method that circumvents point estimates in the output space by directly predicting a normal distribution. used this method along with a particle-based representation for splash prediction. Assuming independent velocity distributions for each splash particle produced visually pleasing . In our initial experiments for stochastic simulations, this method lead to unsatisfactory . We observed that Mean-VarianceEstimation method is capable of indicating highly uncertain situations e.g collisions or force applications. Unfortunately, due to the lack of space-time consistency between particles that are present in real objects, this approach lead to incorrect shape predictions during test time. Stochastic regularization as a way of capturing model uncertainty is an active field of research with scarce theoretical foundations. Nonetheless, we see a growing number of practical applications of this group of algorithms in numerous research areas (; ; ;). proposed applying dropout during training and inference as a Bayesian inference approximation with prediction variance as the measure of the epistemic uncertainty. A clear advantage of this method is the ability to visualize the of each predicted trajectory. On the other hand the computational cost grows linearly with the number of samples. Prior work has shown that injecting noise into neural networks is successful not only as a regularization method but also in the training of RL agents;. In model-free RL, temporal credit assignment, sparse reward, and exploration-exploitation trade-offs present significant challenges. Long episodes amplify both problems of credit assignment and reward sparsity, where naive exploration causes exponentially growing sample inefficiency . Reward shaping is one counter measure that improves credit assignment, and consequently sample efficiency, in model-free RL (Grześ, 2017;). Designing shaping functions usually requires expert knowledge and hand-engineering, while also imposing constraints on how the agent solves the task. Such constraints may prevent the agent from solving the task optimally. Predicting a set of trajectories can be framed here as a reward relaxation method with the clear advantage of depending on a single parameter -dropout rate. introduced parametric noise learned with gradient descent into action prediction network, which led to significantly better exploration and higher rewards without creating large computational overhead. This method shows clear potential for stochastic methods to improve training efficiency in reinforcement learning. Figure 2: Hierarchical Relation Network (HRN) architecture. Force, collision and past effects on particles are computed and then propagated through each object hierarchy. The propagated effects are used to predict the next particle positions. Gray blocks represent graph convolutional effect propagation modules. Our stochastic forward dynamics model is based on the deterministic hierarchical relation network (HRN) as proposed by and depicted in Figure 2. Hierarchical graph representation: Propagating effects through a fully-connected scene graph is computationally infeasible. HRN circumvents this problem by leveraging a tree-like graph form that defines a constrained subset of edges for more efficient effect propagation. Edges within each object graph comprise shape and material properties, describing how rigid, soft and "cloth-like" the material is as defined in our simulator. Edges across object graphs describe physical relations between objects such as contact forces. The hierarchy construction starts with the particles at the lowest level as provided by our ground truth simulator. Each next level of the hierarchy is constructed by clustering particles based on their states. The state attributes are the position, the velocity and the mass. Finally, nodes at each hierarchical level represent the state of an object, an object part, an object subpart and so forth down to single object particles. HRN takes a sequence of the past two hierarchical physics graphs G 1,2 which consist of hierarchical object graphs as input and predicts the next state of the scene graph. Force, Collision and History Modules: HRN assumes three effects that act on particles at the lowest level of the hierarchy and that influence the next particle state: external forces, interactions with particles of other objects in close proximity and particle's state history. Inputs to these modules are current particle states, external forces (Force Module) and states of particles with which the particle interacts (Collision Module). Each module computes an embedding vector that represents the effects of the external forces, the collisions and the past states on the particle using pairwise graph convolutions. These effects are combined by summation and enter the Hierarchical Effect Propagation Module. Hierarchical Effect Propagation Module: HRN predicts the next graph state by estimating the influence that particles within each object and across objects exert upon each other. The compounded effects from the Force, Collision and History Modules are propagated not only up and down the hierarchy, but also between particles at the same level by hierarchical graph convolutions that are implemented as fully-connected networks with weight sharing. Each forward pass takes the states of two connected particles and outputs an embedding vector that describes the influence of the first particle on the latter. These effect embedding vectors are collected for each particle in the scene graph and are used to estimate the future particle velocity. This module is realized as a feed-forward fully-connected neural network and predicts per-particle future velocity. It takes in the current particle state consisting of the position, velocity and mass, as well as the sum of effect embedding vectors belonging to the particle. The predicted particle velocity is expressed in the local coordinate frame, i.e relative the corresponding particle at the higher level in the hierarchy. The state of the particle at the highest level additionally includes gravity and is defined in the global coordinate frame. To make the HRN stochastic and sample multiple plausible trajectories for a given initial state, we introduce a Monte Carlo based dropout on the activations of the graph convolutional collision and force modules. We greatly improve model predictions by introducing a new fully-connected shape loss term and a state-invariant recurrent training procedure. Our final model creates realistic trajectories that are suited to train a model-free agent for manipulation tasks. Stable realistic long-term predictions are crucial for planning tasks. While HRN predictions are accurate for a large number of complex physical scenarios, we identify that objects fall apart relatively quickly along boundaries of object parts for two reasons. First, the HRN's loss function is designed to minimize the error between predicted and ground truth states while imposing a group shape constraint. As shown in the left panel of Figure 3, this group shape loss optimizes the pairwise distances between object nodes within a group to be the same as the ground truth pairwise distances. It does not impose that the pairwise distances across groups are the same as the ground truth distances, which leads to unrealistic deformations between groups as depicted in Figure 5. We therefore introduce a stronger fully-connected shape constraint (Figure Figure 3, right) that imposes pairwise distances to be the same as ground truth pairwise distances across all possible node combinations, which improves shape preservation significantly. Second, the HRN's prediction errors accumulate exponentially as predictions are fed back in recurrently during inference to generate multi-step trajectory predictions. However, during training the HRN is only supervised with the next ground truth state and thus never gets its own perturbed predictions as input. To make the HRN robust against prediction errors, we therefore propose to train the model recurrently in a state-invariant way, i.e. without using a hidden state as physical dynamics is state-free (Figure 4). The overall loss is the sum of losses from each time step. Learning recurrently on long sequences, the network optimizes its weights taking into account its own prediction errors during training. This significantly reduces error accumulation during inference time. Figure 3: Shape loss. The HRN shape loss only constraints particle distances within object particle groups (left). Our new fully-connected shape loss constraints distances between all particles pairs within each objects (right). removes a certain number of randomly chosen nodes in a neural network to prevent overfitting with a probability commonly referred to as dropout rate. In each iteration, a new set of nodes is sampled and only the edge weights attached to the active nodes are updated via backpropagation. In our novel approach, we use randomly sample dropout masks on graph-convolution kernels to sample physically plausible trajectories. To keep the kernel fixed independent of its position in the graph, we only sample once per prediction step. To infer a set of plausible trajectories, we randomly sample a dropout mask for each generated trajectory during test time, similarly how uses dropout to generate multiple predictions. The modular architecture of the HRN allows us to apply dropout at different locations in a very interpretable way (Figure 2 . Dropout on the collision module makes sampled trajectories diverge at collision points. Dropout on the force module leads to diverging trajectories during force applications. Dropout on other HRN modules lead to convergence problems and unrealistic predictions. In the following experiments, we thus only apply dropout to the HRN's force and collision modules. Applying our dropout based sampling method on our dynamics model in physically plausible long-term predictions with consistent shapes. The ability to sample a distribution of physically plausible trajectories can be used to improve the efficiency of exploration of model-free reinforcement learning agents. We thus train a model-free policy on our stochastic physics predictor to achieve physical manipulation tasks as follows. At each episode during training, we input the agent's action and current state into our stochastic forward dynamics predictor and sample a set of 5 future states with our dropout method. For accelerated training, we introduce a reward relaxation method which consists of rewarding an agent as soon as one of the trajectories from the sampled set leads to the goal, naturally exposing the agent to rewards much quicker. If none of the trajectories hits the goal, one future state is chosen at random and the sampling process is repeated for the next future state. The level of reward relaxation is controlled by the dropout rate. The higher the dropout rate, the wider the set of trajectories and the easier it is for the agent to be rewarded. This directs the agent much quicker towards the reward in early training stages. In scenarios requiring high levels of accuracy and repeatability, we find that a gradual reduction of the dropout rate during policy training helps convergence and leads to more efficient policies compared to a fixed dropout rate, which we show in the following experiments. In our experiments, we first show that our recurrent training and new fully-connected shape loss significantly improve the prediction quality for single long-term trajectories on complex physical scenarios over baselines. We then demonstrate how our proposed Monte Carlo sampling based dropout method generates multiple high-quality trajectories by visualizing stochastic model roll-outs. Lastly, we use our stochastic forward dynamics model's ability to generate multiple trajectories to train a model-free policy on two physical manipulation tasks more efficiently and to higher reward. We evaluate our models forward dynamics prediction performance against the HRN baseline on two complex scenarios. The first scenario showcases the ability of our model to predict complex deformations: A deformable soft cube is first lifted off the ground by an upward impulse and then falls toward the ground while rotating and deforming on impact ( Figure 5 left). In the second scenario, we evaluate our models performance on complex collisions. Collisions can greatly magnify object position and pose errors leading to large discrepancies between predictions and ground truth. For our collision experiment, two rigid cubes are placed at a random distance from each other and then repeatedly accelerated by an impulse on each cube into another to generate collisions (Figure 5 right). We train our model on a multitude of examples of both scenarios and evaluate on held out examples. We compare the mean squared error on positions, velocities and shape loss and show qualitative long-term predictions of our model and the HRN baseline. In Table 1, we present our quantitative on the deformation and collision task. Our fullyconnected shape loss and recurrent training procedure significantly lower long-term prediction errors in both scenarios. On the collision task, initial position and velocity error increase slightly compared to the baseline but accumulate to far lower errors in the long run. Empirically, we found that our recurrent training procedure works best with sequence lengths between 4 and 6 time steps. Longer sequence lengths prevent the model from converging during training. We found that gradually increasing the sequence length during training is an effective countermeasure. The improvements of our model over the HRN baseline are obvious in visualizations of predicted trajectories (Figure 5). Whereas HRN predictions fall apart along object boundaries and sometimes penetrate objects, our method preserves shapes and resolves collisions much better and predicts positions much closer to the ground truth, leaving us with an adequate basis for generating multiple plausible trajectories with our sampling method. Figure 5: Dynamics prediction comparisons. Our method is compared to the HRN baseline and ground truth. a) A soft cube bounces of the ground. b) Two rigid cubes collide. Our method preserves the geometry of objects better over long time horizons. In this section, we demonstrate that our Monte Carlo sampling based dropout method can sample multiple physically plausible trajectories from our forward dynamics predictor under the same initial state. In two complex scenarios we study our model's uncertainty during force applications and collisions. In the first scenario, an external force lifts a soft body, which subsequently drops toward the floor rotating slightly. By applying dropout to the force module (Figure 2), we generate multiple trajectories that arise due to our model's uncertainty during force applications. We use a dropout rate of 0.1 during training and 0.3 during testing. In the second scenario, we show that our proposed method produces realistic sets of trajectories in collision scenarios. Forces are applied to two rigid cubes pushing them towards each other causing collision. Dropout is applied to both force and collision module with a rate of 0.05 both during training and testing. The visualizations of multi trajectory roll-outs in Figure 6 show that the sets of predicted trajectories are physically and visually plausible. Due to the modularity of the HRN model, targeted stochasticity can be applied within each submodule via dropout, introducing uncertainty in the output of force and collision predictions. Our proposed sampling method is able to capture trajectory distributions ranging from single mode low variance to complex, multi-modal distributions. Dropout rates between 0.05 and 0.3 allow for fast convergence during training and a wide variety of visually plausible sample trajectories during inference. We notice that inference dropout rates that differ significantly from the training rates can cause biased predictions leading to, e.g. objects slowly drifting away in one direction. Additional studying the effect of the dropout rate on the width of the state distributions can be found in supplement Figure 10. Deformations t t+5 t+23 t+25 t+28 t+33 t+35 t+40 t+45 t+50 t t+1 t+2 t+3 t+4 t+5 t+6 t+7 t+8 t+9 Collisions Falling on a wedge t t+1 t+2 t+3 t+4 t+5 t+6 t+7 t+8 t+9 Multiple materials t t+1 t+2 t+3 t+4 t+5 t+6 t+7 t+8 t+9 Figure 6: Example sampled multiple trajectories. We use dropout on the force and collision module to sample multiple trajectories given the same initial input. Dark colors depict the ground truth trajectory. Light colors depict imagined sampled trajectories. The performance of our model in forward simulation is maintained at a high level despite the introduced graph-convolutional dropout. Deformations: A soft cube bounces off the ground. Collisions: Two rigid cubes collide. Our method is able to sample multiple physically plausible trajectory in each scenario. Falling on a wedge: We simulate the situation from Fig 1. A rigid objects falls on a wedge. Stochastic simulation allows for a multi-modal prediction. Multiple materials: A rigid and a soft object interact with a cloth while falling on the ground. We show that stochastic physical environments are useful for intelligent systems by training reinforcement learning agents in two different scenarios involving various physical interaction types and materials. We use Proximal Policy Optimization as a model-free reinforce-ment learning method in all scenarios. Similarly to , we add a further baseline in which we use the deterministic environment and add Gaussian noise in the action space. Cube moving task: In this scenario, the agent learns to apply a sequence of forces to rigid cubes such that at least one of the cubes is pushed towards the goal region. The maximum length of the episode is 10. Here, the two cubes are transparent to each other and cannot collide. The stochasticity originates entirely from the uncertainty in the force application. We use a constant dropout rate of 0.1 in the force module of our physics predictor throughout the whole training. Ball hitting tower task: Tasks that require inducing collisions pose a significantly more difficult challenge to the RL agent. In this scenario, there is a stack of three rigid cubes and a ball to which the agent can apply force. The agent gets a reward for pushing the middle cube out of the tower. To achieve the goal, the agent needs to hit the tower with the ball to which it applies forces. The maximum episode length is 15 steps. Training update Avg. episode length Figure 7: Average episode length in the "cube moving task". We compare a deterministic environment against action space noise and 2 randomly seeded stochastic environments. The agent learns faster in stochastic environments through better initial exploration and converges to a shorter policy. Training update Avg. episode length Figure 8: Average episode length in the "ballhits-tower task". We compare a deterministic environment against action space noise, a stochastic environment where the dropout rate is fixed and 3 randomly seeded stochastic environments where the dropout rate is annealed. The agent finds shorter policies earlier in the training indicating more efficient exploration in the stochastic environments. Figure 9: Policy comparisons. a) Cube moving task. Top: Policy learned in a deterministic environment (longer, 4 time steps). Bottom: Policy learned in a stochastic environment (shorter, 2 time steps). The first two frames are model inputs. The red cubes indicate the target position to which the green cubes have to be moved. b) Ball hitting tower task. Agents in deterministic and stochastic environments converge to similar 4 step policies. The figure depicts one 4 step policy example. Cube moving task: In this example, introducing the action space noise leads to faster learning compared to learning in the deterministic environment. Training in stochastic physical environments outperforms both baselines, allows for better exploration and finding of shorter policies. In Figure 9, we visualize the two learned policies, in stochastic and deterministic environments. Ball hitting tower task: In this scenario, the agent finds more efficient policies through the application of stronger forces during training, which in gradually shorter policies as shown in Figure 8. The most effective learning method is learning in a stochastic environment with the dropout rate annealing. We lower the dropout rate linearly from 0.1 at the start to 0 after 1200 training updates. This method allows for initial fast exploration, but does not introduce too high stochasticity when precision is needed as the agent begins applying strong forces later in the training. Without annealing, the randomness is too high and agents learn longer policies in a noisy training process, as indicated by the red curve in Figure 8. Furthermore, action space noise improves the pace at which the agent learns compared to the entirely deterministic environment. The policies learned by the presented methods do not significantly differ. An exemplary policy is visualized in Figure 9. Qualitatively our stochastic HRN predicts plausible future trajectories; an experiment in which human subjects were asked to discriminate between ground-truth and predicted trajectories could be used to evaluate its performance quantitatively. Even though this method does not require extensive expert knowledge, a few design decisions have to be made e.g dropout rates for training and inference. During inference, too high of a dropout rate can lead to visually unrealistic dynamics and object interactions. Dropout rate scheduling during training should be investigated to improve convergence of the dynamics model during training, which may improve its performance as an environment for the reinforcement learning tasks. Possible optimizations include more complex, potentially non-linear, annealing schedules during inference, delaying the dropout rate annealing, and finding appropriate starting values. Finding a universal schedule that can be applied to any environment and task has large potential for accelerating reinforcement learning. Further improvements for the physics predictor are key for its use as a physical environment. These can include improvements for: scenarios with multiple materials in one scene, penetrations during collisions that can lead to insufficient position prediction, and generalization to new scenes. Our show that the proposed sampling method produces physically plausible trajectories in single-and multi-object scenarios as well as across a range of materials. The quality of roll-outs, e.g. shape prediction is not compromised by the introduced noise. Furthermore, our model-free reinforcement learning experiments indicate that agents learning in physically stochastic environments are able to explore better and learn quicker, which confirms the quality of the sampled trajectories. In difficult reinforcement learning scenarios, where a high level of precision is needed to get a reward, we demonstrated that dropout rate annealing is an effective method to avoid too high randomness at the same time not reducing the benefits of stochasticity for exploration in early stages of the training. In this regard, stochastic neural physics engines offer a clear advantage over conventional physics engines. In Figure 10, we present the effect of the dropout rate on the width of the predicted distribution of trajectories. In this example, dropout is activated in the force module. We compare distributions generated with dropout rates 0.5, 0.3 and 0.1. In Figure 11, we present the stochastic predictions in "ball hitting tower" scenario. The dropout is active in the force and collision modules. The prediction variance corresponds to the dropout rate strength in both scenarios. In the "ball hitting tower" scenario, we observe highly complex behavior, where the time at which the cubes fall off the tower at different rates

Represent the short summary for finding the corresponding scientific article (long)

We propose a stochastic differentiable forward dynamics predictor that is able to sample multiple physically plausible trajectories under the same initial input state and show that it can be used to train model-free policies more efficiently.

represent this text to find a much much longer article about it with say hundreds-thousands of words

Inspired by trial-to-trial variability in the brain that can result from multiple noise sources, we introduce variability through noise in the knowledge distillation framework and studied their effect on generalization and robustness.