Research Subject 1: Interaction of Awareness

Nakazawa Group

There are moments when people need information. For example, when an action is needed to escape a crisis, or when an action is needed to maintain health, or when an action to have fun. People may or may not know that such behaviors exist. To address these issues, researchers specializing in information systems, including our group, suppose to display such behaviors on smartphones and computers. It would give people more opportunities to take better actions. Nakazawa’s group also takes this strategy, but people often do not behave as indicated. If they are in a hurry, have other interests, or are in a slightly bad mood, it is common for these personal states to keep them from taking action.

Today’s software behaves the same way for all people when interacting with them. Although the displayed information is possibly optimized based on a person’s log, the same information is displayed in the same manner and at the same time. If the software know more about a person’s personal state and behaved accordingly with this state, it can deliver information more deeply to a person’s mind. A calming color scheme for someone who is in a hurry, a stronger attention-grabbing expression for someone with other interests, or a little later information for someone in a bad mood could be considered, respectively. The ability of software to change its informational behavior according to a person’s personal state is important in considering the symbiosis between software including ML model and people.

Smartphones and smart watches are equipped with a variety of sensors that can detect blood pressure, heart rate, body temperature, and transportation. From the detected raw values, various estimations can also be carried out, for example, calculating the means of transportation, such as walking or driving, from the values of acceleration sensors, for example. However, estimation of personal state has not been realized except in some cases, and software has not yet demonstrated its full potential. For example, personal state such as personality, short-term moods and mood fluctuations, physical conditions including headaches and PMS, hunger and fatigue, etc., greatly influence people behavior. These states are therefore important for future software to understand. In addition, the effective way for software to change its own behavior based on the understanding of these states has not yet been clarified. Although there are various factors that can be changed, such as the timing of information presentation, the format of information presentation, and the wording of information, it is necessary to clarify the effects of these changes.

Therefore, the Nakazawa Group is conducting research on ① technology to better estimate person’s personal states and ② information presentation technology adaptive to a person’s personal state, especially using wearable devices such as smartphones and smart watches. The purpose of these studies is to make people happy through the power of information. If we can better convey to people information that can make them healthier, safer, and more enjoyable, the power of information will be more strongly demonstrated, and in some way it will make people happier. “Smart city” is a city in which more such information flows, and in this sense, this research can be said to be about making the city smarter. “Smart city” is not just a city with many computers, networks, and sensors installed, but a city where a lot of powerful information is circulating.


Technology to Estimate Users’ Personal States

A wide range of data is constantly collected in smartphones and wearable devices that include physical sensors such as GPS and soft sensors that output estimated transportation states. Explanatory variables such as personal logs and service usage logs obtained from cloud services and ground truth label obtained from self-reported answer value of users’ personal states during data collection experiments are learned to estimate users’ demographics, personality, organizational states and psychological states. These factors include static states such as gender, age, and personality, as well as dynamic states such as physical condition and dependency level.

Technology for Adaptive Information Presentation to Users’ Personal State

When suggesting action to a user, it can be expressed in an emotional way through images and sounds, or in a logical and explanatory way through language and numerical values. Depending on the personal states of a use who receive the information, sometimes the emotional way is effective, sometimes the logical way is effective. Whichever representation way is used, the content of the information can be decided as the effect of doing the action, the loss of not doing the action, and the difference between the effect and the loss. In this study, we focus largely on the control of expression and content, and also examine the details of expression and content, such as tone of voice and word phrasing.


  • c-SNE: Emotion-based Deep Cross-modal Retrieval by using Stochastic Neighbor Embedding

    Cross-modal retrieval based on subjective information aims to enable flexible media retrieval services, such as allowing users to specify, for example, a text or/and an image to search audio clips. The resulting audio clips should have an impression similar to the specified text/image. Existing methods focus on building cross-media cross-modal relationships using objective information from…

  • Self-Control Methods for SNS Use Using Discomfort Caused by Reduced Usability

    With the spread of social networking services, excessive use of these services has become a problem. It is also known as SNS addictions and cause various adverse physical and mental health effects. Restricting the use of SNS has become a very difficult task because various reward systems incorporated in SNS and the high convenience of…

  • Large-scale Estimation of People’s Moods using Web Search Queries and Mobile Sensor Data

    Focusing on the emotional state which is closely related to the rationality states, we constructed machine learning models to estimate the emotional state (mood score) from the user’s search keywords by combining (1) values obtained from various sensors on smartphones, (2) self-reported subjective data obtained through the Experience Sampling Method (ESM), and (3) search keywords…

  • Estimation of Personality Traits using News Browsing Logs

    Although the number of articles published on news portal sites has been increasing in recent years, there is a growing demand for appropriate news recommendations to improve the user experience. In this paper, we focus on user’s personality traits and propose a method to estimate personality traits only from news browsing logs collected from actual…

  • Attention Management: AI Estimates Optimal Timing for Push Notifications

    In today’s ubiquitous computing environment where users carry, manipulate, and interact with an increasing number of networked devices, applications and web services, human attention is the new bottleneck in computing. It is therefore important to minimize a user’s mental effort due to notifications, especially in situations where users are mobile and using multiple wearable and…

  • Detecting Emotional Eating with Mobile Computing

    Eating behavior influenced by stress and emotional changes is called emotional eating. The reasons of emotional eating are autonomic nervous system and hormone imbalance due to stress and lifestyle disturbance. Therefore, it is possible to predict the occurrence of emotional eating by continuously observing and analyzing the activity level of the autonomic nervous system and…

  • SmileWave: Investigating the Occurrence of Selfie-based Emotional Contagion over Social Network

    Happiness is obviously one of the most fundamental essence that affects many aspects of our lives.Past research found that happiness of one person affect that of other people. What occurs under this propagation of emotion is called “emotional contagion”, a phenomenon wherein through perception, people experience the same emotion expressed by someone when communicating with…

  • SFC GO: A system to Support Building Connections between Students in Online Physical Education Classes

    Due to the spread of COVID-19, all classes in the spring semester of 2020 at Keio University, including physical education (PE), was held online. Even though “Physical Education 1” which all first year student have to take, is generally an opportunity for classmates to communicate with each other through physical exercise. Therefore it was necessary…