User:WillWare/Automation of science

Notes from a slide show

Computers should

• look for patterns in data (data mining)
• propose falsifiable hypotheses
• design experiments to test hypotheses
• perform experiments & collect data
• confirm/deny hypotheses
• mine new data for new patterns, repeat

Precedents

Eurisko

Doug Lenat and Eurisko gained notoriety by submitting the winning fleet (a large number of stationary, highly weaponed, defenseless ships) to the United States Traveller TCS national championship in 1981, forcing extensive changes to the game's rules.

This story has circulated quite a bit and is quite famous. Unfortunately Lenat has never released the source code for Eurisko. So an open-source version of Eurisko would be a welcome thing, and that idea was an early stimulus of my thinking about automation of science.

Adam the "Robot Scientist"

Reported in April 2009 by Ross King at Aberystwyth University. It uses lab automation to perform experiments, and data mining to find patterns in the resulting data. Adam developed novel genomics hypotheses about S. cerevisiae yeast and tested them. Adam's conclusions were manually confirmed by human experimenters, and found to be correct.

Eureqa

• Website

Eureqa is a software tool for detecting equations and hidden mathematical relationships in your data. Its primary goal is to identify the simplest mathematical formulas which could describe the underlying mechanisms that produced the data. Eureqa is free to download and use, but AFAICT it is not open source. So we need an open source equivalent. Luckily the ideas behind Eureqa are laid out pretty plainly.

Eureqa takes a data set and generates a curve-fitting function for that data. Genetic programming appears to be the preferred way for doing this. Hod Lipson discussed Eureqa in his talk, and his comment was that it's easy to generate a model but hard to generate an interpretation, a story that explains why that model is the right one.

What next?

Adam is designed to work alone. No connection to the broader scientific literature. Adam is confined to one very narrow problem domain. To broaden the effort, we need

• International standard, an ontology for machine-parseable sharing of scientific reasoning processes
  • data sets
  • hypotheses
  • predictions
  • deduction, induction, statistical inference
  • design of experiments
• versions of Adam designed for other problem domains
• some amount of shared vocabulary, otherwise each works in isolation

Reasoning scenarios

• Pure symbolic logic (no probabilities orconfidence levels)
  • Semantic web, inference engines, first order logic
• Hypotheses with blanket probabilities
  • each hypothesis describes a world, each world has logic propositions, but no probabilities
  • use empirical evidence to update blanket probabilities
• Assign probabilities to individual propositions
  • Statistical inference replaces logical deduction
  • Get smart about the role of uncertainty
• Work with noisy analog data
  • Get smart about signal processing, probability distributions
  • Study the noise to look for deeper structures

Semantic markup for existing scientific and medical literature

Immediately useful for constructing a semantic search engine for medicine and research

Motivates development of science ontology

Machines should eventually publish journal articles

Maybe it will tell us something interesting about how humans do science

Fund long-term work by monetizing near-term work

IANAVC, but maybe one of these would work...

• Semantic search engine for doctors and researchers
• Build an oracle, win bets - politics, finance, climate
• Dual-license it and charge for commercial use
• Offer consulting services