Natural language understanding (NLU) aims to allow computers to understand text automatically. NLU may seem easy to humans, but it is extremely difficult for computers because of the variety, ambiguity, subtlety, and expressiveness of human languages. Recent efforts to NLU have been largely exemplified in tasks such as natural language inference, reading comprehension and question answering. A common practice is to pre-train a language model such as BERT on large corpora to learn word representations and fine-tune on task-specific data. Although BERT and its successors have achieved state-of-the-art performance in many NLP tasks, it has been found that pre-trained language models mostly only reason about the surface form of entity names and fail to capture rich factual knowledge. Moreover, NLU models built on such pre-trained language models are susceptible to adversarial attack that even a small perturbation of an input (e.g., paraphrase questions and/or answers in QA tasks) would result in dramatic decrease in models' performance, showing that such models largely rely on shallow cues. In human reading, successful reading comprehension depends on the construction of an event structure that represents what is happening in text, often referred to as the situation model in cognitive psychology. The situation model also involves the integration of prior knowledge with information presented in text for reasoning and inference. Fine-tuning pre-trained language models for reading comprehension does not help in building such effective cognitive models of text and comprehension suffers as a result. In this fellowship, I aim to develop a knowledge-aware and event-centric framework for natural language understanding, in which event representations are learned from text with the incorporation of prior background and common-sense knowledge; event graphs are built on-the-fly as reading progresses; and the comprehension model is self-evolved to understand new information. I will primarily focus on reading comprehension and my goal is to enable computers to solve a variety of cognitive tasks that mimic human-like cognitive capabilities, bringing us a step closer to human-like intelligence.

Natural language understanding (NLU) aims to allow computers to understand text automatically. NLU may seem easy to humans, but it is extremely difficult for computers because of the variety, ambiguity, subtlety, and expressiveness of human languages. Recent efforts to NLU have been largely exemplified in tasks such as natural language inference, reading comprehension and question answering. A common practice is to pre-train a language model such as BERT on large corpora to learn word representations and fine-tune on task-specific data. Although BERT and its successors have achieved state-of-the-art performance in many NLP tasks, it has been found that pre-trained language models mostly only reason about the surface form of entity names and fail to capture rich factual knowledge. Moreover, NLU models built on such pre-trained language models are susceptible to adversarial attack that even a small perturbation of an input (e.g., paraphrase questions and/or answers in QA tasks) would result in dramatic decrease in models' performance, showing that such models largely rely on shallow cues. In human reading, successful reading comprehension depends on the construction of an event structure that represents what is happening in text, often referred to as the situation model in cognitive psychology. The situation model also involves the integration of prior knowledge with information presented in text for reasoning and inference. Fine-tuning pre-trained language models for reading comprehension does not help in building such effective cognitive models of text and comprehension suffers as a result. In this fellowship, I aim to develop a knowledge-aware and event-centric framework for natural language understanding, in which event representations are learned from text with the incorporation of prior background and common-sense knowledge; event graphs are built on-the-fly as reading progresses; and the comprehension model is self-evolved to understand new information. I will primarily focus on reading comprehension and my goal is to enable computers to solve a variety of cognitive tasks that mimic human-like cognitive capabilities, bringing us a step closer to human-like intelligence.

The ecosystem of compute devices is highly connected, and likely to become even more so as the internet-of-things concept is realized. There is a single underlying global protocol for communication which enables all connected devices to interact, i.e. internet protocol (IP). In this project, we will create a corresponding single underlying global protocol for computation. This will enable wireless sensors, smartphones, laptops, servers and cloud data centres to co-operate on what is conceptually a single task, i.e. an AnyScale app. A user might run an AnyScale app on her smartphone, then when the battery is running low, or wireless connectivity becomes available, the app may shift its computation to a cloud server automatically. This kind of runtime decision making and taking is made possible by the AnyScale framework, which uses a cost/benefit model and machine learning techniques to drive its behaviour. When the app is running on the phone, it cannot do very complex calculations or use too much memory. However in a powerful server, the computations can be much larger and complicated. The AnyScale app will behave in an appropriate way based on where it is running. In this project, we will create the tools, techniques and technology to enable software developers to create and deploy AnyScale apps. Our first case study will be to design a movement controller app, that allows a biped robot with realistic humanoid limbs to 'walk' over various kinds of terrain. This is a complex computational task - generally beyond the power of embedded chips inside robotic limbs. Our AnyScale controller will offload computation to computers on-board the robot, or wirelessly to nearby servers or cloud-based systems. This is an ideal scenario for robotic exploration, e.g. of nuclear disaster sites.

In recent years, the trust that society places in opaque centralised mechanisms run by government, network operators, and financial institutions has been eroding, with various events (e.g., the financial meltdown of 2007 and the hack of the DigiNotar certificate authority) illustrating that high integrity cannot be achieved merely through trust in one or a handful of parties. As a reaction to this erosion in trust, two alternative architectures have emerged: users have either flocked to systems that have no central point of trust; or they have increased pressure on central entities to provide more openness and visibility. In both of these settings, the main technique that has emerged to provide these properties is a distributed ledger; i.e., a list of events that have occurred within a given system that is created and stored by a distributed or even decentralised set of parties. Storing such ledgers in a distributed and transparent manner allows these systems to achieve full public auditability, in which any user can check for themselves that the system is functioning correctly. Given the potential applications of distributed ledgers, one might be tempted to use a single approach as a way to provide auditability or distribute trust. Requirements in one setting may be very different from those in another, however, so one approach cannot be indiscriminately applied. As an example, SSL certificates are public, so their issuance can be stored on a public ledger. On the other end of the spectrum, systems such as financial settlement, supply chains, and personal identity management all deal with highly sensitive data that cannot be included as-is in a globally visible ledger. Balancing these application-specific requirements with both the benefits and limitations of distributed ledgers is the main focus of our research. To understand the requirements in each of the settings mentioned above, our research will be conducted with five user partners: the Bank of England, which is interested in using distributed ledgers for financial settlement; the Department of Work and Pensions (DWP), which is interested in the provision of benefits; the Robin Hood Fund, which is interested in allowing for the trading of entitlements to the fund; Provenance, which is interested in transparency in supply chain certification; and the Google Certificate Transparency team, which is already using distributed ledgers to log the issuance of SSL certificates. Each of these user partners will give us insight into a different potential application of distributed ledgers, and by constructing technical solutions that meet their diverse requirements (e.g., the need for privacy or scalability), we can impact their eventual deployments of these technologies.

Security systems break because design practices focus too much on mechanisms, at the expense of clearly-defined properties. The vision of this research is to bring about a shift of emphasis to highlight the properties that security systems are expected to provide. This will be done by developing methods for verification of security systems. I will focus on a selection of interconnected real-world problems that are of great importance to society, but that are currently in need of greater industry/academe cooperation. The combination of fundamental research with close collaboration with industry, government and users is expected to achieve significant results and impact. I will develop and apply new methods and techniques to create and analyse solutions in three areas:* Trusted computing is an industry-led technology that aims to root security in hardware. Since its launch, academics including me have discovered significant issues that threaten to undermine its potential at providing a range of security benefits. This has arisen because industry does not have the expertise to analyse the protocols.* Electronic voting is an application currently attracting significant interest from government and industry, but numerous security issues have resulted in failure of confidence among politicians, commentators and public alike.* Privacy for citizens using electronic services is hotly debated by journalists and user groups and politicians, but has been substantially eroded by new technologies and policies.In these three areas, there is currently the risk of significant waste of resources on inappropriate or unaccepted technologies, resulting in user disempowerment and exclusion. The outcomes of this fellowship are intended to address that risk.A distinguishing feature of the proposal is the substantial engagement with industry and user groups that are active in these three areas. As a result of discussions with them, several organisations have committed significant resources, including cash contribution, manager and developer time, and access to users and experts.