Beyond the Polanyi Paradox: How Human-Aware AI Unlocks Creativity And Breakthrough Discoveries

Artificial intelligence (AI) has transformed scientific research and innovation, yet as previously discussed, the Polanyi Paradox remains a significant obstacle to using AI to generate creative solutions or breakthrough discoveries. According to the Polanyi Paradox, AI models that only learn from explicit knowledge will lack the tacit or intuitive knowledge of humans and may generate obvious results. Hence, approaches to addressing the Polanyi Paradox are of interest to anyone using AI models to seek new discoveries. 

Human-aware AI models, as reported by Sourati and Evans in Nature Human Behaviour, offer an approach to developing AI models that are more effective in generating creative solutions and discoveries – or even scientific breakthroughs. The human-aware AI model developed by the authors improved AI prediction of new material science discoveries by 400%. 

The human-aware AI model was fed scientific data and authorship information to enable measurement of the “distribution of human scientists around each topic involved in the candidate discoveries.” By using this meta-information, the human-aware AI models can “mimic” the intuition or tacit knowledge of scientists by predicting and avoiding the “human crowd.” In other words, the human-aware AI models do not have to rely solely on structured data, but can:

1. Simulate how scientists approach discovery rather than just predicting outcomes.
2. Use scientific heuristics to refine AI-driven experimentation.
3. Generate complementary “alien” hypotheses. Unlike traditional AI models that mostly work within known research spaces, human-aware AI models are designed to identify connections between seemingly unrelated fields and bridge gaps that scientists might take years to discover.
4. Prioritize high-value experiments over trivial optimizations and balance exploration vs. exploitation to ensure breakthrough discoveries.

Therefore, human-aware AI models have many advantages to ensure that they not only generate predictable and obvious solutions but also complementary alien hypotheses that can lead to breakthrough discoveries and more creative solutions. While this approach will not resolve the Polanyi Paradox, it could at least help create future AI models that are less prone to generating obvious solutions because they lack the tacit knowledge of humans. 

Our analysis underscores the power of incorporating human and social factors to produce artificial intelligence that complements rather than substitutes for human expertise. In accounting for not just human expertise, but the complete distribution of scientific experience and exposure, such systems can be designed to race with rather than against the scientific community, expanding the scope of human imagination and discovery.