Our Vision

We are a team of scientists and engineers working together to solve key challenges that the world is facing. We focus on using machine learning to optimize complex decision making and reduce inefficiencies.

What We Do

At present we are successfully deploying our machine learning techniques to optimize the electricity grid by improving advance planning of electricity production and distribution, resulting in decreased economic waste and environmental damage.

Our Impact

Reduced CO2 emissions and pollution.
Improved reliability of the grid.
Less waste and increased economic efficiency.

OPTIMIZING THE ELECTRICAL GRID

The electrical grid is one of the most complex systems ever created.

We use machine learning to improve the operational planning of electricity production and distribution, improving reliability, efficiency, transparency, and reducing pollution.

problem
Electricity cannot be efficiently stored. Power lines and transformers have limited capacity to carry power. Equipment on the grid can fail unexpectedly. Power generation is not scheduled accurately. Power use is not accurately anticipated. Electricity supply and demand must be balanced at all times.
 
implications
Power lines get congested when nearing thermal limits, which means that power to serve some location must be sourced from more distant or expensive generators. Sourcing power from alternative sources to meet the unexpected demand is expensive and can be operationally risky. Tapping into environmentally unfriendly sources leads to increased greenhouse gas emissions and pollution that is harmful to human health.
 
impact
Increased reliability of the grid by helping create a better plan for one of the most complex systems ever created. Decreased greenhouse gas emissions and harmful pollution. Reduced waste and improved economic efficiency leading to lower electricity rates for those most in need.
 

What We Do

ELECTRICAL GRID CHALLENGES
Electricity cannot be economically stored at utility scales. Supply and demand must be balanced at all times, and the structure of the transmission leads to massive amounts of inefficiency. In order to deal with these issues, and to ensure reliable access to power, the global standard is to centralize control under System Operators, who coordinate the grid for electric utilities. The efficiency of the electricity grid is also of fundamental importance for achieving lower carbon emissions, and reducing the impact of coal and natural gas pollution on human health.
WASTE AND POLLUTION
Electricity is the source of 25% of global CO2 emissions and billions of dollars in economic waste. The US electricity sector alone emits 2 billion tons of CO2 yearly, accounting for 38% of the country's total energy related CO2 emissions (2013)٭, power plants were responsible for 64% of SO2 emissions,16% of NOX emissions, 40% of CO2 emissions, and 68% of mercury air emissions in the US. The human health impacts are on par with traffic accidents.
WHAT WE DO
Already successfully deployed from coast to coast in North American electricity grids, Invenia is actively growing and looking at expanding electrical grid optimization work globally. We interact directly with the grids, helping to plan for generation, flow and use of electricity in advance of real time operations. We help the system operators to optimize the power grid to ensure reliability, efficiency, transparency, while reducing harmful emissions.

Focus Areas

Energy Systems
Electricity cannot be economically stored at utility scales. Supply and demand must be balanced at all times, and the structure of the transmission grid massive amounts of inefficiency. In order to deal with these issues, and to ensure reliable access to power, the global standard is to centralize control under Independent System Operators, who coordinate the grid for electric utilities. The efficiency of the electricity grid is also of fundamental importance for achieving lower carbon emissions, and reducing the impact of coal pollution on human health. Understanding all aspects of the electrical grid and their properties and complexities is an area of our core focus at present.
Machine Learning
Invenia's machine learning is focused on systems with a fundamental importance to everyday life, but which are not dealt with efficiently. Our central interest is in optimizing complex decision-making and resource usage under uncertainty. In particular, the electricity grid offers data with unique properties, including many time series with complex structures, and operations that change rapidly. While our focus is on practical problems, we are also actively working on more fundamental research with a focus on AI, time series forecasting and decision-making.
Development, Architecture and Operations
At Invenia, software development expertise is applied to building and optimizing our distributed machine learning system: the Energy Intelligence System (EIS). Developers at Invenia also build and support the underlying systems and foundational technical tools, focusing on long-term solutions while also addressing current needs and anticipating future requirements. The success and versatility of these systems allow Invenia to tackle increasingly complex problems, while expanding its horizons and capabilities.
Complex Systems
Complex Systems is a broad area of research focused on studying systems made of a large number of interacting components and the emergence of complex collective behaviors. Social systems, the brain, and electricity grids are examples. A related area of interest is Complex Adaptive Systems, referring to systems reorganizing themselves to solve complex problems. Neural Networks and Memristors are two such examples. We are focused on the practical aspects in the application of these tools, but also advancing the fields from a theoretical perspective.
Finance
Our interest in finance tools and techniques stems from our work in electricity grids, which provide unique challenges with transmission due to the physical properties of electricity. We are studying risk, the processes that improve the efficiency of the grid and the consequences of regulatory changes. There are also many interesting connections between finance, statistical mechanics and complex systems to explore.

Join Us

Being a part of the Invenia team presents an opportunity to work with and learn from amazing people with expertise in machine learning, theoretical physics, mathematics, complex systems, and computer science while contributing to research that has a positive impact on our society and the environment.

We find great purpose in our ability to change the world for the better. It's what drives us to work hard and continuously improve. If our vision resonates with you and you are interested in joining us, please visit our career page at
www.joininvenia.com to apply.

Our Team

We are a team of scientists, researchers and developers that come from machine learning, engineering, computer science, economics, theoretical physics, mathematics and management.
Leadership
Advisors
Architecture and Operations
Development
Research
Finance
People Operations
Talent Acquisition
Chief Executive Officer / Co-Founder
Matt Hudson
Matt co-founded Invenia, and has been CEO from the start. He started Invenia while at Microsoft, after majoring in political science and economics with additional studies in computer science and engineering. He has developed a deep knowledge of the electrical grid, complex networks, and machine learning.
Chief Technology Officer / Co-Founder
Christian Steinruecken
Christian completed his PhD under the supervision of Prof Sir David MacKay at the University of Cambridge (UK) and is a specialist in machine learning. He has led engineering projects in artificial intelligence, data compression, and probabilistic programming. Christian believes that building intelligent technology is our best hope for making the world a better place.
Chief People Operations Officer / Co-Founder
Oksana Koval
Oksana is a co-founder of Invenia and is currently the Chief People Operations Officer, overseeing operations in Canada and the UK. She attended the University of Manitoba where she studied Anthropology as a post-graduate. In her spare time, she also pursues research interests in machine learning and archaeology. In the past, she has excavated a 3,000-year-old settlement in Crete, Greece, and for her Master’s thesis, she applied machine learning to identify the ceramic manufacturing techniques of ancient potters.
Managing Director Invenia Labs/ Chief Science Officer / Co-Founder
Cozmin Ududec
Cozmin is a co-founder of Invenia and now also the Managing Director and Chief Science Officer of Invenia Labs in Cambridge. He received his PhD in the foundations of quantum theory from the University of Waterloo and is still puzzling over the quantum world in his spare time.
Scientific Advisor / Co-founder
David Duvenaud
David is a co-founder of Invenia and an assistant professor in computer science and statistics at the University of Toronto. He received his PhD in machine learning from Cambridge University. Before Invenia, he also worked at Google Research, the Max Planck Institute for Intelligent Systems, and the Harvard Intelligent Probabilistic Systems group.
Researcher
Abraham Alvarez-Bustos
Abraham Alvarez-Bustos was born in Mexico. He received a B.Sc. degree in Electrical Engineering from Instituto Politécnico Nacional (IPN) - ESIME, Mexico City, Mexico, in 2012. Then, a M.Sc. degree in Electrical Power Systems from the IPN - Sección de Estudios de Posgrado e Investigación (SEPI), Mexico City, Mexico, in 2015, where he got First Class Honours and an Honorific Mention. In 2016 he worked as a Transmission System Analyst in the National Control Centre of Energy (CENACE), Mexico. He obtained a Ph.D. degree at Durham University, Durham, UK in the area of Power Systems optimization and Control. His principal research interests lie in developing methods, models and software aimed at Power Systems Computational Analysis and Optimisation for Planning.
Researcher
Alex Robson
Alex has a PhD in Biophysics from Oxford University, where he worked on applying machine learning techniques to model biological data. Since then, he's worked on numerous applications of ML, including one for a startup in the energy sector and another for risk models in fintech. In his spare time, Alex can often be found playing board games or occasionally hacking around on personal ML projects.
Researcher
Andrew Rosemberg
Andrew received a BSc in control engineering from PUC-RIO and a BSc in general engineering from École Centrale de Marseille. He also holds a master's in Electrical Engineering with an emphasis on Operation Research, focusing on Power Systems and Energy markets. Some of his previous projects revolve around energy economic dispatch analysis and simulation, financial data classification and portfolio optimization. His main interests are optimization, decisions under uncertainty and machine learning.
Researcher
Anton Isopoussu
Most recently, Anton has been interested in unsupervised learning using ideas from optimal transport. Before getting into machine learning, he studied mathematics, physics and computer science in Helsinki and then went on to complete a PhD in algebraic geometry in Cambridge.
Developer
Arnaud Henry
Arnaud graduated from the University of Edinburgh with an MSc in Artificial Intelligence in 2012. Since then, he's been working at various companies as a software engineer, notably in the music streaming industry and in the emerging space of autonomous vehicles. In his spare time, he enjoys travelling and tinkering with pet projects.
Researcher
Astrid Dahl
After achieving a Masters in Economics and Econometrics at the University of Sydney, Astrid completed her PhD in machine learning at the University of New South Wales. She has previously worked as a professional econometrician in the energy and financial sectors but now, as a researcher for Invenia, she finds ways of improving the computational efficiency of multi-task Gaussian process models for solar power forecasting. Her main research interests are scalable nonparametric methods for spatiotemporal modeling, structured prediction and grid integration of distributed generation.
Software Developer
Bailey Shirtliff
Bailey graduated in 2018 with a BSc in Computer Science from the University of Manitoba. He is an enthusiastic Python developer, with previous experience working on cloud-native software. His hobbies include beatboxing, painting miniatures, and playing board/card/video games.
Power Systems Researcher
Bashar Anwar
Muhammad Bashar Anwar received his BEng degree in Electrical Engineering in 2013 from the University of Hong Kong and a MSc in Electrical Power Engineering from Masdar Institute of Science and Technology (Khalifa University), UAE in 2015. He then went on to complete his PhD in Electrical Engineering from University College Dublin, Ireland in 2019. His research interests include developing optimization and equilibrium models for power systems operation, investments, electricity markets, and grid integration of renewables through sector-coupling, electric vehicles, and demand response.
Senior Researcher - Forecast Track Lead
Bella Wu
Bella achieved her PhD in engineering at the University of Cambridge, where she developed advanced signal processing techniques, including many based on Bayesian inference, for magnetic resonance applications. Before joining Invenia, Bella worked at a startup on building energy models that provide forecast and analysis for use in hedging, trading, and investments. She is interested in combining mathematical modelling and machine learning with fundamental theories in fields such as engineering and economics to gain unique insights into complicated systems that have a significant social impact.
People Operations Associate
Bianca Felisbino
Bianca is originally from Brazil and moved to Canada in 2017. She has a bachelor’s degree in Chemical Engineering and a postgraduate diploma in Environmental Management and Sustainability. She has experience in technical and leadership roles in the manufacturing and hospitality industries. Throughout her career, she discovered her passion for sustainability, and for supporting and encouraging the core of every company: its people. She loves outside activities, nature, and getting to know different cultures and flavors.
Research Engineer
Branwen Snelling
Branwen did her PhD in Geophysics at Imperial College London, where she used an array of computational fluid dynamics models to research landslide-tsunami hazards. Along the way she explored uncertainty quantification and machine learning methods for natural hazard assessment. She is interested in building and using modelling tools to better understand the natural world and other complex systems.
Developer
Brendan Curran-Johnson
Brendan is a developer and a documentarian at Invenia. Whether he's writing docs or infrastructure code, his work is integral for others to be able to do theirs.
Developer
Cameron Ditchfield
Cameron originally joined Invenia as a co-op student from the University of Manitoba. An enthusiastic reader, Cameron enjoys a wide range of topics. From the sagas to The Guns of August, he likes to spend his free time with a good book.
Personal and Executive Assistant to the CEO
Camilla Regan
After graduating from Oxford Media and Business School, Camilla began her career working in London. Before joining Invenia she has worked as a Personal Assistant for various high profile individuals in different industries.
People Operations Associate
Caolán Jennings
Caolán has been a Cambridge local since 2007. With an academic background in philosophy and law, he is always ready for a friendly chat. Outside of work you can normally find him enjoying a good book or playing music with friends.
Senior Researcher - Group Lead for Data Science
Chris Davis
Chris was formerly an Assistant Professor in Energy Informatics and Modelling at the University of Groningen. He received his PhD at the Delft University of Technology, and his work covers topics related to Energy, Sustainability, Linked Data, Machine Learning, Data Visualisation, and Agent-Based Modelling. Here are a few papers published by Chris: 1. Secondary Resources in the Bio-Based Economy: A Computer Assisted Survey of Value Pathways in Academic Literature 2. Electric vehicle charging in China’s power system: Energy, economic and environmental trade-offs and policy implications 3. The state of the states: Data-driven analysis of the US Clean Power Plan For the rest of Chris' published work, please refer to his Google Scholar profile.
Developer
Cole Peters
Cole graduated from the University of Manitoba in 2019 with a Bachelor of Computer Science (Honours) degree. He started working at Invenia as a co-op student in 2018 and transitioned to a full-time position after graduation. Outside of work, Cole enjoys hiking, camping, reading, and gaming.
Chief Financial Officer
Dan Allen
Dan is an experienced financial leader who has held senior positions in several tech businesses. His corporate finance experience spans from raising debt and equity funds to leading on several mergers and acquisitions deals, including exits. Commercially he has experience of building scalable operational and financial infrastructure in global tech businesses and has led operational and data teams. He has vast experience of being a member of strategic management teams, including holding both exec and non-exec board seats.
Advisor
Doyne Farmer
Alongside his work with Invenia as a Research Advisor, Doyne is also a Professor in the Mathematical Institute at the University of Oxford and an External Professor at the Santa Fe Institute. His current research is in economics, including agent-based modeling, financial instability, and technological progress. He was a founder of Prediction Company, a quantitative automated trading firm that was sold to the United Bank of Switzerland in 2006. His past research includes complex systems, dynamical systems theory, time series analysis, and theoretical biology.
Head of Architecture
Eric Davies
Eric received a Bachelor's Degree in Computer Science with a specialisation in Artificial Intelligence from the University of Manitoba. Now, Eric architects and develops improvements to Invenia's data processing and machine learning pipeline while pushing for a faster, more capable EIS. Eric also contributes to the Julia community and helps lead the charge for new technologies at Invenia.
Researcher - Decision Track Lead
Eric Perim Martins
Eric received his PhD in Physics from the University of Campinas, where he worked on Nanotechnology problems using Computational Physical-Chemistry techniques. He then moved to Duke University where he looked into High-Throughput Materials Science methods before joining Invenia.
Senior Developer
Fernando Chorney
Fernando is a huge fan of trying to make things easier and more efficient for all people and systems. When he’s not messing with his Linux servers at home, Fernando likes to cook, make music, play games and learn new skills.
Research Engineer - Group Lead for Research Software Engineering
Frames Catherine White
Frames is passionate about building the tools to do research better. Her current specific interests are around tooling for machine learning libraries, in particular automatic differentiation. Frames completed her PhD on Natural Language Processing via Machine Learning (Titled “On the surprising capacity of linear combinations of embeddings”) at the University of Western Australia in 2018. She has been programming Julia since 2014, and is currently a maintainer of, and contributor to, some unreasonable number of packages.
Research Advisor
Francesco Caravelli
Francesco's research focuses on statistical physics as well as complex systems, in particular, complex networks, memristive circuits, econophysics, and agent-based modelling. He is a theoretical physicist, interested in quantum and classical systems and the application of techniques of statistical physics and complexity to other disciplines such as economics, engineering, and finance. He has been a Senior Researcher at Invenia Labs in Cambridge and a researcher at the London Institute for Mathematical Sciences, before moving as an Oppenheimer Fellow to Los Alamos National Laboratory.
Research Engineer
Glenn Moynihan
Glenn Moynihan is native of county Cork in Ireland but upped sticks to study Theoretical Physics at Trinity College Dublin where he received his Bachelor's in 2013 and his PhD in 2018. His research focused on improving the accuracy and scope of linear-scaling density-functional theory applied to large-scale calculations of materials exhibiting strongly-correlated electrons.
Researcher
Ian Goddard
Ian is a recent Master's graduate in data science from the University of Edinburgh where he has previously completed a Bachelor's degree in Physics. His main area of interest is how we can use data science and machine learning to help in the transition of the energy sector to a more renewable mix. His interests in machine learning are broad, with a particular interest in probabilistic modelling, Bayesian learning, and graph theory.
People Operations Associate
Jackie Diab
Jackie graduated in 2021 with a CIPD certified BSc in Business and Human Resource Management from Anglia Ruskin University. Before joining Invenia she has worked in various people management roles within the retail industry and has experience with HR administration, recruitment and delivering training. She’s passionate about fostering a healthy culture in the workplace, and is excited by the emergent field of people analytics.
Developer
James Hamblin
James graduated with a BSc in Mathematics from Loughborough University. Before working at Invenia, he worked as a DevOps engineer for a startup developing artificial intelligence software for the legal profession.
Senior Researcher
James Requeima
James is a senior researcher at Invenia and a PhD student studying machine learning at the University of Cambridge in the Computational and Biological Learning Lab, under the supervision of Dr. Richard Turner. His interests include Bayesian optimisation, learning, approximate inference methods, and deep generative models. He previously completed a master’s in machine learning, speech, and language technology at the University of Cambridge under the supervision of Dr. Zoubin Ghahramani and a master’s of mathematics from the McGill University under Daniel Wise.
Talent Specialist
Jasmin Geddes-Rainbow
After gaining a degree in languages from the University of Cambridge, Jasmin spent a number of years working in business operations management at the University before choosing to specialize in Talent Acquisition. Outside of work, she can usually be found singing with her band, travelling, or in the gym.
People Operations Manager
João Moraes
After completing a law degree Joao went into business through a variety of roles, ultimately completing an MBA at the University of Cambridge. His previous background includes managing all aspects of a restaurant chain, brand strategy consultancy, and leadership development.
Data Scientist
Josh Chadney
Josh is a space and atmospheric physicist with a PhD in planetary aeronomy from Imperial College London, where he worked on building physical models of the upper atmosphere of extrasolar planets. He has since performed measurements of the northern lights from Svalbard, in the high Arctic, to infer the temperature of Earth’s upper atmosphere as a postdoctoral research fellow at the University of Southampton. More recently he has built systems to measure and monitor gas flaring and pollutant emissions produced by the energy industry for an environmental services start-up.
Accounting Associate
Kristan Aho
Kristan moved to Winnipeg to obtain her bachelor's degree in Environmental Studies. Soon after accomplishing that, she found herself on a long journey pursuing her CPA designation. Kristan has a wide range of hobbies, from gardening to camping to playing D&D with friends. When she isn't crunching numbers or crushing fictional mythical beasts, she can be found with her two boys, husband and furry family members.
Researcher
Letif Mones
Letif received his PhD in computational chemistry from ELTE University, Hungary, where he worked on the methodological improvement of hybrid QM/MM approaches and free energy computations for complex systems. At the University of Cambridge and later at the University of Warwick, he developed an efficient sampling technique in combination with Gaussian process regression. In addition to this, he also introduced a protocol for constructing high dimensional quantum surfaces of organic molecules using machine learning and developed universal preconditioners to enhance the performance of optimisation techniques.
Payroll and Finance Operations Associate
Lisa Morris
Lisa was born and raised in Calgary, Alberta, and has lived in Winnipeg since 2005. She has a diploma in Human Resource Management from the University of Winnipeg and an undergraduate degree in Criminology (Sociology/Psychology) from the University of Manitoba. In her spare time, Lisa volunteers with a local animal rescue, enjoys going to concerts, Jets games, Goldeyes games, and spending time with her partner and their two dogs.
Data Engineer
Lukas Timmerman
Lukas graduated from the University of Manitoba in 2020 with a BSc in Computer Science, specializing in Theoretical Computer Science, and Networks and Security. Outside of work, Lukas spends time producing music, creating art, and developing video games.
Facilities Coordinator
Magda Karavatou
Magda was born in Thessaloniki, Greece, and moved to Cambridge in 2019. She has previously studied Economic Science at the University of Macedonia and has received an IT skills certificate from the University of Cambridge.
Researcher - Group Lead for Power Systems
Mahdi Jamei
Mahdi Jamei received a BSc degree in Science and Technology in 2013 and an MSc in Electrical and Computer Engineering from Florida International University in 2014. He then went on to achieve a Ph.D. in Electrical and Computer Engineering from Arizona State University in 2018. His research interests lie primarily in developing computational analytic tools for power systems employing mathematical and signal processing techniques. He has been studying the security challenges of interdependent critical infrastructures of power, cyber and gas networks over the past few years.
Power Systems Engineering
Mana Jalali
Mana received her bachelor’s degree in Electrical engineering from the University of Tehran. She then moved to the US and got her master’s and PhD in electrical engineering at the University of Virginia Tech. Her dissertation revolved around applying Gaussian processes and Bayesian optimization to power system applications, such as inferring dynamic oscillations and strategic investment in electricity markets.
Developer
Mary Jo Ramos
Mary Jo graduated with a BSc in Genetics before realizing her true passion for Computer Science. Before joining Invenia, she gained experience in Web Development and Bioinformatics. In her spare time, she loves to explore restaurants and shops around the city, discover the latest in technology and fashion, and play the Sims.
Software Developer
Matthew Brzezinski
In 2017, Matt graduated from the University of Manitoba with a BSc in Computer Science. His experience so far has mainly been in WebDev, DevOps, and systems architecture. He is currently very interested in data analytics, learning more about systems design, and contributing to StackOverflow. Outside of software development, he loves playing video games, tinkering with his car, going to AutoCross and is slowly working towards his Time Attack license. He also loves to cook and learn about new cuisines (currently learning more Thai recipes).
Data Scientist - Emissions Track Lead
Max Lensvelt
Max has worked in power markets and renewable energy within the UK, Europe, and the United States for almost a decade. His experience encompasses analytical roles in private equity, utilities, and industry. Most recently, he has worked as a Data Scientist for the developer of the UK's first virtual power plant. He also holds a BSc in Physics and an MSc in Data Science.
Data Scientist
Meas Meng
Meas was previously a postdoctoral researcher at the University of California, Santa Barbara. She received her PhD in environmental engineering from the University of Southern California. She also holds an MS in mechanical engineering from USC, and a BS in mechanical engineering from California State University, Los Angeles.
People Operations Associate
Michelle Davis
Michelle is from Winnipeg. She has worked in different industries throughout her career, including transportation, education, and health care, in both private and public sectors. Michelle is currently in the Human Resource Management program at the University of Manitoba. In her spare time she loves to read, hike, and play pickleball.
Research Engineer
Miha Zgubič
Miha, originally from Slovenia, is a theoretical physicist by training and holds an MSci degree from Imperial College London. After completing his Masters, he moved to the countryside to pursue a PhD at the University of Oxford where he analysed the data from the Large Hadron Collider at CERN searching for a rare decay of the Higgs boson. He likes python, boosted decision trees, and Federer's one-handed backhand.
Head of Operations
Mike de Denus
While working on his Master's degree, Mike developed a robotics system for maintaining formation movement with varying numbers of robots without the use of a centralized controller. His teams have won awards at numerous international robotics competitions. At Invenia, he focuses on the analysis and exploration of nodal and spot electricity markets.
Head of Development
Nick Thiessen
After completing a BSc in computer science, Nick came to Invenia to work on building and maintaining machine learning systems and simulations. During his spare time, he can be found either developing, playing, or discussing games of all sorts.
Researcher
Raphael Saavedra
Raphael received his MSc and BSc degrees in Electrical Engineering with an emphasis in Operations Research from the Pontifical Catholic University of Rio de Janeiro. He previously worked in projects with major electricity distribution companies with the goal of forecasting demands and optimizing contracts. He is also a Julia programmer and contributes to open- source projects. His main interests are optimization, time series models, and power system operation.
People Operations Manager
Reena Varshney
Reena is a CPHR (Chartered Professional in Human Resources) professional, with skills in advisory HR, employee relations and innovative work solutions.Reena's interest in HR is transformative over transactional. Work that is strategic, forward-thinking, and proactive excites her. Reena also strives to bring a humanistic approach to HR and to always consider the 'why' behind what people do.  In her spare time she enjoys baking, lifting in the gym, travel and reading.
Talent Specialist
Robbie Smith
Robbie graduated from the University of Southampton with a Bachelor’s degree in Music, where he specialised in composition. Since then, he has worked in technical recruitment and a variety of administrative roles. Robbie loves learning about new technologies and computing. He also enjoys writing music and comedy in his spare time.
Research Engineer
Rory Finnegan
Rory Finnegan joined Invenia as a Computer Science co-op and Linux enthusiast with a background in Bioinformatics and Human-Computer Interactions. In 2016, Rory completed a Master’s degree in Computational Neuroscience and is now working towards his PhD.
Data Engineer
Ryan Froese
After originally joining Invenia as a co-op student, Ryan achieved his Bachelor’s Degree in Computer Science at the University of Manitoba. He spends his free time exploring new technologies, bike riding and playing video games.
Developer - Backbone Track Lead
Sam Massinon
Sam is a recent graduate from the University of Manitoba with a Bachelor of Computer Science. He joined Invenia as a co-op during the summer of 2015 and started working full time in December that year. Since then, he has been involved in several projects ranging from development to researching.
Developer
Sam Morrison
Sam has recently finished her Bachelor of Computer Science (Honours) degree at the University of Manitoba. When not at work or studying, she can be found tending to plants, watching science fiction shows, and trying her hand at new hobbies and skills.
Researcher - Group Lead for Machine Learning
Sean Lovett
Sean received his PhD in computational physics from the University of Cambridge, where he worked on adaptive meshing methods in computational fluid mechanics. Since then, he has worked as a research scientist in R&D for the oil and gas industry, where his research interests included complex fluids, reduced-order models, and the application of statistical modelling to physical systems. He has also been involved in several academic collaborations and physics outreach projects.
People Operations Associate
Shenya Wickramasinghe
Shenya originally hails from Sri Lanka and spent her early years growing up in Dubai. She completed her bachelor’s degree in Switzerland at the Cesar Ritz Colleges where she graduated with a Bachelor of Arts in Hospitality Business Management. After working within various roles in the hospitality industry, Shenya moved to Winnipeg in 2019 to pursue her passion for human resources. She has recently obtained a postgraduate diploma in Human Resources Management from the University of Winnipeg. In her spare time, Shenya enjoys watching documentaries, painting and baking.
Machine Learning Researcher
Simon Nakach
Simon Nakach has a PhD in Theoretical physics from Imperial College, where he studied how symmetry can lead to the next theory of particle Physics. After his PhD he spent two and a half years working on a range of Machine Learning projects ranging from Computer Vision to time series forecasting as a Data Scientist. He is interested in leveraging physically-motivated machine learning techniques to better understand and model real world systems
Finance Senior Manager
Stephen Maharaj
Stephen is a Chartered Professional Accountant (CPA), Chartered Accountant (CA) with experience in audit, financial reporting, operations, and tax compliance. After starting his career at Deloitte, Stephen transitioned to industry and held Controller roles with Uncharted Software and Harvest One. Outside of the office you can find Stephen travelling the world, checking out live events, art galleries, or exploring the latest lounges and restaurants with family and friends. 
Director of Finance
Steve Marr
Steve is a Chartered Accountant who has previously worked at Deloitte, Great-West Life and most recently as the Corporate Controller for the International Institute for Sustainable Development. He strives for continuous improvement and enjoys problem-solving collaboratively.
Developer
Timothy Levins
Timothy was born in Malaysia and has recently graduated from the University of Manitoba with a bachelor’s of science degree in Computer Science. His work at Invenia started as part of his first internship as a junior developer, and he spent most of his time working with the data feeds team to build Invenia’s data gathering framework. In his spare time, he loves to explore new places.
Machine Learning Research Intern
Tomisin Dada
Tomisin recently completed the MPhil in Advanced Computer Science at the University of Cambridge. He has previously worked as a Project Engineer in Nigeria and completed his Bachelors in Electrical Engineering at Imperial College. Tomisin believes Machine Learning has a large role to play in building an equitable, secure and sustainable power grid. In his spare time Tomisin enjoys music, reading and writing poetry.
Developer
Tyler Loewen
Tyler completed his BSc in Computer Science at the University of Manitoba in 2021 with a focus in Software Engineering, Databases, and HCI & Graphics. In his free time he enjoys photography, riding motorcycles, hiking, and the outdoors.
Researcher
Wessel Bruinsma
Wessel is currently a PhD student at the University of Cambridge. He holds an MPhil in Machine Learning and Machine Intelligence also from the University of Cambridge. At Invenia, Wessel researches machine learning and investigates applications thereof. His main research interests include probabilistic modelling, Bayesian nonparametrics, approximate inference, and signal processing.
Research Associate
Will Tebbutt
Will is currently a PhD student in the Machine Learning Group at the University of Cambridge. He is interested in probabilistic models for spatio-temporal phenomena, in particular Gaussian processes. He occasionally advises on specific machine learning problems at Invenia. When not working, he can be found playing the guitar, or listening to people play it well.
Machine Learning Researcher
Will Wilkinson
Will received his PhD in Computer Science from Queen Mary University of London in 2019, after which, he spent 2.5 years as a postdoctoral researcher in Machine Learning at Aalto University, Finland. His research interests lie at the intersection of machine learning and signal processing, with a focus on statistical inference for spatio and spectro-temporal data using Gaussian processes.
Software Engineer
Wynand Badenhorst
Wynand graduated in 2019 with a BSc in Computer Science, specializing in Computer Systems, Human-Computer Interaction and Graphics. Before joining Invenia, his experience has been in full stack development of web and desktop applications, as well as development of hybrid cloud applications. In his free time he likes to watch sci-fi movies, play video games, build things, cook things, and make things in general.
Advisor
Zoubin Ghahramani
Zoubin works with Invenia as an advisor. He is also a professor of Information Engineering at the University of Cambridge, where he leads the Machine Learning Group consisting of about 30 researchers, and the Cambridge Liaison Director of the Alan Turing Institute, the UK's national institute for Data Science. His academic career includes concurrent appointments as one of the founding members of the Gatsby Computational Neuroscience Unit in London, and as a faculty member of CMU's Machine Learning Department for over ten years. His current research interests include statistical machine learning, Bayesian nonparametrics, scalable inference, probabilistic programming, and building an automatic statistician. He has published over 250 papers, receiving over 30,000 citations (an h-index of 74).

Open Source

At Invenia we want to support the community by open sourcing various projects we work on.

Below is a list of various projects that are open to the world! Here's a blog post on how to contribute to open source too!

https://www.invenia.ca/wp-content/uploads/AWS-logo-test-01-1.png
AWS.jl

A Julia interface for Amazon Web Services.

https://www.invenia.ca/wp-content/uploads/Chain-rules-Smaller03-01.png
ChainRules.jl

The ChainRules package provides a variety of common utilities that can be used by downstream automatic differentiation (AD) tools to define and execute forward-, reverse-, and mixed-mode primitives.

https://www.invenia.ca/wp-content/uploads/Chain-rules-Smaller03-01.png
ChainRulesCore.jl

The ChainRulesCore package provides a light-weight dependency for defining sensitivities for functions in your packages, without you needing to depend on ChainRules itself.

https://www.invenia.ca/wp-content/uploads/2016/08/Julia_logo2.png
ExprTools.jl

 ExprTools provides tooling for working with Julia expressions during metaprogramming. This package aims to provide light-weight performant tooling without requiring additional package dependencies.

https://www.invenia.ca/wp-content/uploads/2016/08/Julia_logo2.png
FeatureTransforms.jl

FeatureTransforms.jl provides utilities for performing feature engineering in machine learning pipelines with support for AbstractArrays and Tables.

https://www.invenia.ca/wp-content/uploads/2016/08/Julia_logo2.png
Intervals.jl

Define intervals with left and round bounds.

https://www.invenia.ca/wp-content/uploads/full-logo-1.png
LibPQ.jl

LibPQ.jl is a Julia wrapper for the PostgreSQL libpq C library.

https://www.invenia.ca/wp-content/uploads/2016/08/Julia_logo2.png
Mocking.jl

Allows Julia function calls to be temporarily overloaded for the purpose of testing.

https://www.invenia.ca/wp-content/uploads/2016/08/Julia_logo2.png
NamedDims.jl

NamedDimsArray is a zero-cost abstraction to add names to the dimensions of an array.

Papers

Invenia's researchers work on numerous interesting topics.

This occurs both within the company, and with collaborators beyond.

In the spirit of open scholarship, below we have some papers for you to explore!

All
https://www.invenia.ca/wp-content/uploads/22Leveraging-power-grid-topology-in-machine-learning-assisted-optimal-power-flow22.png
Leveraging power grid topology in machine learning assisted optimal power flow
Published at IEEE-TPS. Machine learning assisted optimal power flow (OPF) aims to reduce the computational complexity of these non-linear and non-convex constrained optimization problems by consigning expensive (online) optimization to offline training. The majority of work in this area typically employs fully connected neural networks (FCNN). However, recently convolutional (CNN) and graph (GNN) neural networks have also been investigated, in effort to exploit topological information within the power grid. Although promising results have been obtained, there lacks a systematic comparison between these architectures throughout literature. Accordingly, we introduce a concise framework for generalizing methods for machine learning assisted OPF and assess the performance of a variety of FCNN, CNN and GNN models for two fundamental approaches in this domain: regression (predicting optimal generator set-points) and classification (predicting the active set of constraints). For several synthetic power grids with interconnected utilities, we show that locality properties between feature and target variables are scarce and subsequently demonstrate marginal utility of applying CNN and GNN architectures compared to FCNN for a fixed grid topology. However, with variable topology (for instance, modeling transmission line contingency), GNN models are able to straightforwardly take the change of topological information into account and outperform both FCNN and CNN models.
Thomas Falconer, Letif Mones
https://www.invenia.ca/wp-content/uploads/PRACTICAL-CONDITIONAL-NEURAL-PROCESSES-VIA-TRACTABLE-DEPENDENT-PREDICTIONS.png
Practical Conditional Neural Processes Via Tractable Dependent Predictions
Conditional Neural Processes (CNPs; Garnelo et al., 2018a) are meta-learning models which leverage the flexibility of deep learning to produce well-calibrated predictions and naturally handle off-the-grid and missing data. CNPs scale to large datasets and train with ease. Due to these features, CNPs appear well-suited to tasks from environmental sciences or healthcare. Unfortunately, CNPs do not produce correlated predictions, making them fundamentally inappropriate for many estimation and decision making tasks. Predicting heat waves or floods, for example, requires modelling dependencies in temperature or precipitation over time and space. Existing approaches which model output dependencies, such as Neural Processes (NPs; Garnelo et al., 2018b) or the FullConvGNP (Bruinsma et al., 2021), are either complicated to train or prohibitively expensive. What is needed is an approach which provides dependent predictions, but is simple to train and computationally tractable. In this work, we present a new class of Neural Process models that make correlated predictions and support exact maximum likelihood training that is simple and scalable. We extend the proposed models by using invertible output transformations, to capture non-Gaussian output distributions. Our models can be used in downstream estimation tasks which require dependent function samples. By accounting for output dependencies, our models show improved predictive performance on a range of experiments with synthetic and real data.
Stratis Markou, James Requeima, Wessel P. Bruinsma, Anna Vaughan, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/Modelling-Non-Smooth-Signals-with-Complex-Spectral-Structure.png
Modelling Non-Smooth Signals with Complex Spectral Structure
The Gaussian Process Convolution Model (GPCM; Tobar et al., 2015a) is a model for signals with complex spectral structure. A significant limitation of the GPCM is that it assumes a rapidly decaying spectrum: it can only model smooth signals. Moreover, inference in the GPCM currently requires (1) a mean-field assumption, resulting in poorly calibrated uncertainties, and (2) a tedious variational optimisation of large covariance matrices. We redesign the GPCM model to induce a richer distribution over the spectrum with relaxed assumptions about smoothness: the Causal Gaussian Process Convolution Model (CGPCM) introduces a causality assumption into the GPCM, and the Rough Gaussian Process Convolution Model (RGPCM) can be interpreted as a Bayesian nonparametric generalisation of the fractional Ornstein-Uhlenbeck process. We also propose a more effective variational inference scheme, going beyond the mean-field assumption: we design a Gibbs sampler which directly samples from the optimal variational solution, circumventing any variational optimisation entirely. The proposed variations of the GPCM are validated in experiments on synthetic and real-world data, showing promising results.
Wessel P. Bruinsma, Martin Tegnér, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/Wide-Mean-Field-Bayesian-Neural-Networks-Ignore-the-Data-1.png
Wide Mean-Field Bayesian Neural Networks Ignore the Data
Bayesian neural networks (BNNs) combine the expressive power of deep learning with the advantages of Bayesian formalism. In recent years, the analysis of wide, deep BNNs has provided theoretical insight into their priors and posteriors. However, we have no analogous insight into their posteriors under approximate inference. In this work, we show that mean-field variational inference entirely fails to model the data when the network width is large and the activation function is odd. Specifically, for fully-connected BNNs with odd activation functions and a homoscedastic Gaussian likelihood, we show that the optimal mean-field variational posterior predictive (i.e., function space) distribution converges to the prior predictive distribution as the width tends to infinity. We generalize aspects of this result to other likelihoods. Our theoretical results are suggestive of underfitting behavior previously observered in BNNs. While our convergence bounds are non-asymptotic and constants in our analysis can be computed, they are currently too loose to be applicable in standard training regimes. Finally, we show that the optimal approximate posterior need not tend to the prior if the activation function is not odd, showing that our statements cannot be generalized arbitrarily.
Beau Coker, Wessel P. Bruinsma, David R. Burt, Weiwei Pan, Finale Doshi-Velez
https://www.invenia.ca/wp-content/uploads/Machine-learning-assisted-industrial-symbiosis-Testing-the-ability-of-word-vectors-to-estimate-similarity-for-material-substitutions.jpeg
Machine learning-assisted industrial symbiosis: Testing the ability of word vectors to estimate similarity for material substitutions
A challenge of facilitating industrial symbiosis involves identifying novel uses of waste streams that can satisfy the demands of other industries. For these efforts, a variety of characteristics must often be considered. A mine of relevant knowledge has been gathered in resources such as academic journals and patent databases. However, in looking to harness the potential of such data to support facilitation, compiling information on expansive ranges of material properties and technical requirements from a variety of unstructured sources can pose a significant manual effort. To ameliorate this, we demonstrate and evaluate an automated system that, given a large collection of patents and academic articles related to waste valorization, is able to assist with the process of identifying which waste streams could potentially be used as substitute feedstocks. Instead of aiming to measure (potentially thousands of) material properties directly, we use word correlations as a proxy to reflect “common knowledge.” Novel in furthering this approach is the application of word vectors, which have emerged as a promising natural language processing tool. The process employs a machine learning approach where words are represented as high-dimensional vectors which encode latent features related to words that often appear around it. When this approach is assessed by comparing its suggestions to documented cases, the use of vectors shows potential to incorporate latent information in data-based explorations. Further research into how this approach compares, and could be integrated with, established symbiosis development practices will be key to understanding its full potential and drawbacks.
Chris Davis, Graham Aid
https://www.invenia.ca/wp-content/uploads/AbstractDifferentiation.jl-Backend-Agnostic-Differentiable-Programming-in-Julia.png
AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia
(Best Poster Award)
 
No single Automatic Differentiation (AD) system is the optimal choice for all problems. This means informed selection of an AD system and combinations can be a problem-specific variable that can greatly impact performance. In the Julia programming language, the major AD systems target the same input and thus in theory can compose. Hitherto, switching between AD packages in the Julia Language required end-users to familiarize themselves with the user-facing API of the respective packages. Furthermore, implementing a new, usable AD package required AD package developers to write boilerplate code to define convenience API functions for end-users. As a response to these issues, we present AbstractDifferentiation.jl for the automatized generation of an extensive, unified, user-facing API for any AD package. By splitting the complexity between AD users and AD developers, AD package developers only need to implement one or two primitive definitions to support various utilities for AD users like Jacobians, Hessians and lazy product operators from native primitives such as pullbacks or pushforwards, thus removing tedious -- but so far inevitable -- boilerplate code, and enabling the easy switching and composing between AD implementations for end-users.
Frank Schäfer, Mohamed Tarek, Lyndon White (Frames Catherine White), Chris Rackauckas
https://www.invenia.ca/wp-content/uploads/Screen-Shot-2022-06-14-at-8.14.05-PM.png
Assessing the Cost of Network Simplifications in Long-Term Hydrothermal Dispatch Planning Models
The sustainable utilization of hydro energy relies on accurate estimates of the opportunity cost of the water. This value is calculated through long-term hydrothermal dispatch problems (LTHDP), and the recent literature has raised awareness about the consequences of modeling simplifications in these problems. The inaccurate representation of Kirchhoff's voltage law under the premise of a DC power flow is an example. Under a non-linear AC model, however, the LTHDP becomes intractable, and the literature lacks an accurate evaluation method of different modeling alternatives. In this paper, we extend the state-of-the-art cost-assessment framework of network approximations for LTHDP and bring relevant and practical new insights. First, we increase the quality of the assessment by using an AC power flow to simulate and compare the performance of five policies based on different network approximations. Second, we find that the tightest network relaxation (based on semidefinite programming) is not the one exhibiting the best performance. Results show that the DC power flow with quadratic losses approximation exhibits the lowest expected cost and inconsistency gaps. Finally, its computational burden is lower than that exhibited by the semidefinite relaxation, whereas market distortions are significantly reduced in comparison to previously published benchmarks based on DC power flow.
Andrew W. Rosemberg; Alexandre Street; Joaquim Dias Garcia; Davi M. Valladão; Thuener Silva; Oscar Dowson
https://www.invenia.ca/wp-content/uploads/How-Tight-Can-PAC-Bayes-be-in-the-Small-Data-Regime_.png
How Tight Can PAC-Bayes be in the Small Data Regime?
In this paper, we investigate the question: Given a small number of datapoints, for example N = 30, how tight can PAC-Bayes and test set bounds be made? For such small datasets, test set bounds adversely affect generalisation performance by withholding data from the training procedure. In this setting, PAC-Bayes bounds are especially attractive, due to their ability to use all the data to simultaneously learn a posterior and bound its generalisation risk. We focus on the case of i.i.d. data with a bounded loss and consider the generic PAC-Bayes theorem of Germain et al. While their theorem is known to recover many existing PAC-Bayes bounds, it is unclear what the tightest bound derivable from their framework is. For a fixed learning algorithm and dataset, we show that the tightest possible bound coincides with a bound considered by Catoni; and, in the more natural case of distributions over datasets, we establish a lower bound on the best bound achievable in expectation. Interestingly, this lower bound recovers the Chernoff test set bound if the posterior is equal to the prior. Moreover, to illustrate how tight these bounds can be, we study synthetic one-dimensional classification tasks in which it is feasible to meta-learn both the prior and the form of the bound to numerically optimise for the tightest bounds possible. We find that in this simple, controlled scenario, PAC-Bayes bounds are competitive with comparable, commonly used Chernoff test set bounds. However, the sharpest test set bounds still lead to better guarantees on the generalisation error than the PAC-Bayes bounds we consider.
Andrew Y. K. Foong, Wessel P. Bruinsma, David R. Burt, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/The-Gaussian-Neural-Process.png
The Gaussian Neural Process
Neural Processes (NPs; Garnelo et al., 2018a,b) are a rich class of models for meta-learning that map data sets directly to predictive stochastic processes. We provide a rigorous analysis of the standard maximum-likelihood objective used to train conditional NPs. Moreover, we propose a new member to the Neural Process family called the Gaussian Neural Process (GNP), which models predictive correlations, incorporates translation equivariance, provides universal approximation guarantees, and demonstrates encouraging performance.
Wessel P. Bruinsma, James Requeima, Andrew Y. K. Foong, Jonathan Gordon, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/Deep-learning-architectures-for-inference-of-AC-OPF-solutions.png
Deep learning architectures for inference of AC-OPF solutions
Machine learning assisted optimal power flow (OPF) aims to reduce the computational complexity of these non-linear and non-convex constrained optimization problems by consigning expensive (online) optimization to offline training. The majority of work in this area typically employs fully connected neural networks (FCNN). However, recently convolutional (CNN) and graph (GNN) neural networks have also been investigated, in effort to exploit topological information within the power grid. Although promising results have been obtained, there lacks a systematic comparison between these architectures throughout literature. Accordingly, we introduce a concise framework for generalizing methods for machine learning assisted OPF and assess the performance of a variety of FCNN, CNN and GNN models for two fundamental approaches in this domain: regression (predicting optimal generator set-points) and classification (predicting the active set of constraints). For several synthetic power grids with interconnected utilities, we show that locality properties between feature and target variables are scarce and subsequently demonstrate marginal utility of applying CNN and GNN architectures compared to FCNN for a fixed grid topology. However, with variable topology (for instance, modeling transmission line contingency), GNN models are able to straightforwardly take the change of topological information into account and outperform both FCNN and CNN models.
 
Poster
Thomas Falconer, Letif Mones
https://www.invenia.ca/wp-content/uploads/WEmbSim-A-Simple-yet-Effective-Metric-for-Image-Captioning.png
WEmbSim: A Simple yet Effective Metric for Image Captioning
(DSTG Best Contribution to Science Award)
 
The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements. Based on our past attempts at developing highly sophisticated learning-based metrics, we have discovered that a simple cosine similarity measure using the Mean of Word Embeddings (MOWE) of captions can actually achieve a surprisingly high performance on unsupervised caption evaluation. This inspires our proposed work on an effective metric WEmbSim, which beats complex measures such as SPICE, CIDEr and WMD at system-level correlation with human judgments. Moreover, it also achieves the best accuracy at matching human consensus scores for caption pairs, against commonly used unsupervised methods. Therefore, we believe that WEmbSim sets a new baseline for any complex metric to be justified.
Naeha Sharif, Lyndon White (Frames Catherine White), Mohammed Bennamoun, Wei Liu, Syed Afaq Ali Shah
https://www.invenia.ca/wp-content/uploads/Meta-Learning-Stationary-Stochastic-Process-Prediction-with-Convolutional-Neural-Processes.png
Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes
Stationary stochastic processes (SPs) are a key component of many probabilistic models, such as those for off-the-grid spatio-temporal data. They enable the statistical symmetry of underlying physical phenomena to be leveraged, thereby aiding generalization. Prediction in such models can be viewed as a translation equivariant map from observed data sets to predictive SPs, emphasizing the intimate relationship between stationarity and equivariance. Building on this, we propose the Convolutional Neural Process (ConvNP), which endows Neural Processes (NPs) with translation equivariance and extends convolutional conditional NPs to allow for dependencies in the predictive distribution. The latter enables ConvNPs to be deployed in settings which require coherent samples, such as Thompson sampling or conditional image completion. Moreover, we propose a new maximum-likelihood objective to replace the standard ELBO objective in NPs, which conceptually simplifies the framework and empirically improves performance. We demonstrate the strong performance and generalization capabilities of ConvNPs on 1D regression, image completion, and various tasks with real-world spatio-temporal data.
Andrew Y. K. Foong, Wessel P. Bruinsma, Jonathan Gordon, Yann Dubois, James Requeima, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/WordTokenizers.jl-Basic-tools-for-tokenizing-natural-language-in-Julia.png
WordTokenizers.jl: Basic tools for tokenizing natural language in Julia
WordTokenizers.jl is a tool to help users of the Julia programming language (Bezanson, Edelman, Karpinski, & Shah, 2014) work with natural language. In natural language processing (NLP) tokenization refers to breaking a text up into parts – the tokens. Generally, tokenization refers to breaking a sentence up into words and other tokens such as punctuation. Complementary to word tokenization is sentence segmentation or sentence splitting (occasionally also called sentence tokenization), where a document is broken up into sentences, which can then be tokenized into words. Tokenization and sentence segmentation are some of the most fundamental operations to be performed before applying most NLP or information retrieval algorithms. WordTokenizers.jl is currently being used by packages like TextAnalysis.jl, Transformers.jl and CorpusLoaders.jl for tokenizing text.
Ayush Kaushal , Lyndon White (Frames Catherine White), Mike Innes , Rohit Kumar
https://www.invenia.ca/wp-content/uploads/Scalable-Exact-Inference-in-Multi-Output-Gaussian-Processes.png
Scalable Exact Inference in Multi-Output Gaussian Processes
Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling O(n3p3 ), which is cubic in the number of both inputs n (e.g., time points or locations) and outputs p. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to O(n3m3 ). However, this cost is still cubic in the dimensionality of the subspace m, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in m in practice, allowing these models to scale to large m without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.
Wessel Bruinsma, Eric Perim, William Tebbutt, Scott Hosking, Arno Solin, Richard Turner
https://www.invenia.ca/wp-content/uploads/Square-Reduction-of-the-Optimal-Power-Flow-Problem-through-Meta-Optimization-1.png
Reduction of the Optimal Power Flow Problem through Meta-Optimization
  We introduce a method for solving Optimal Power Flow (OPF) using meta-optimization, which can substantially reduce solution times. A pre-trained classifier that predicts the binding constraints of the system is used to generate an initial reduced OPF problem, defined by removing the predicted non-binding constraints. Through an iterative procedure, this initial set of constraints is then ex- tended by those constraints that are violated but not represented in the reduced OPF, guaranteeing an optimal solution of the original OPF problem with the full set of constraints. The classifier is trained using a meta-loss objective, defined by the computational cost of the series of reduced OPF problems.
Poster
Alex Robson, Mahdi Jamei, Cozmin Ududec, Letif Mones
https://www.invenia.ca/wp-content/uploads/Square-GP-ALPS-Automatic-Latent-Process-Selection-for-Multi-Output-Gaussian-Process-Models.png
GP-ALPS: Automatic Latent Process Selection for Multi-Output Gaussian Process Models
A simple and widely adopted approach to extend Gaussian processes (GPs) to multiple outputs is to model each output as a linear combination of a collection of shared, unobserved latent GPs. An issue with this approach is choosing the number of latent processes and their kernels. These choices are typically done manually, which can be time consuming and prone to human biases. We propose Gaussian Process Automatic Latent Process Selection (GP-ALPS), which automatically chooses the latent processes by turning off those that do not meaningfully contribute to explaining the data. We develop a variational inference scheme, assess the quality of the variational posterior by comparing it against the gold standard MCMC, and demonstrate the suitability of GP-ALPS in a set of preliminary experiments. AABI_Poster
Pavel Berkovich, Eric Perim, Wessel Bruinsma
https://www.invenia.ca/wp-content/uploads/Square-Learning-an-Optimally-Reduced-Formulation-of-OPF-through-Meta-optimization.png
Learning an Optimally Reduced Formulation of OPF through Meta-optimization
With increasing share of renewables in power generation mix, system operators would need to run Optimal Power Flow (OPF) problems closer to real-time to better manage uncertainty. Given that OPF is an expensive optimization problem to solve, shifting computational effort away from real-time to offline training by machine learning techniques has become an intense research area. In this paper, we introduce a method for solving OPF problems, which can substantially reduce solve times of the two-step hybrid techniques that comprise of a neural network with a subsequent OPF step guaranteeing optimal solutions. A neural network that predicts the binding status of constraints of the system is used to generate an initial reduced OPF problem, defined by removing the predicted non-binding constraints. This reduced model is then extended in an iterative manner until guaranteeing an optimal solution to the full OPF problem. The classifier is trained using a meta-loss objective, defined by the total computational cost of solving the reduced OPF problems constructed during the iterative procedure. Using a wide range of DC- and AC-OPF problems, we demonstrate that optimizing this meta-loss objective results in a classifier that significantly outperforms conventional loss functions used to train neural network classifiers. We also provide an extensive analysis of the investigated grids as well as an empirical limit of performance of machine learning techniques providing optimal OPF solutions.
Alex Robson, Mahdi Jamei, Cozmin Ududec, Letif Mones
https://www.invenia.ca/wp-content/uploads/Convolutional-Conditional-Neural-Processes.png
Convolutional Conditional Neural Processes
We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data. Translation equivariance is an important inductive bias for many learning problems including time series modelling, spatial data, and images. The model embeds data sets into an infinite-dimensional function space as opposed to a finite-dimensional vector space. To formalize this notion, we extend the theory of neural representations of sets to include functional representations, and demonstrate that any translation-equivariant embedding can be represented using a convolutional deep set. We evaluate ConvCNPs in several settings, demonstrating that they achieve state-of-the-art performance compared to existing NPs. We demonstrate that building in translation equivariance enables zero-shot generalization to challenging, out-of-domain tasks.
Gordon, J., Bruinsma W. P., Foong, A. Y. K., Requeima, J., Dubois Y., Turner R. E.
https://www.invenia.ca/wp-content/uploads/Square-Meta-Optimization-of-Optimal-Power-Flow.png
Meta-Optimization of Optimal Power Flow
The planning and operation of electricity grids is carried out by solving various forms of con- strained optimization problems. With the increasing variability of system conditions due to the integration of renewable and other distributed energy resources, such optimization problems are growing in complexity and need to be repeated daily, often limited to a 5 minute solve-time. To address this, we propose a meta-optimizer that is used to initialize interior-point solvers. This can significantly reduce the number of iterations to converge to optimality. Poster
Mahdi Jamei, Letif Mones, Alex Robson, Lyndon White (Frames Catherine White), James Requeima, Cozmin Ududec
https://www.invenia.ca/wp-content/uploads/Square-Memristive-networks-From-graph-theory-to-statistical-physics.png
Memristive networks: From graph theory to statistical physics
This paper is an introduction to a very specific toy model of memristive networks, for which an exact differential equation for the internal memory which contains the Kirchhoff laws is known. In particular, we highlight how the circuit topology enters the dynamics via an analysis of directed graph. We try to highlight in particular the connection between the asymptotic states of memristors and the Ising model, and the relation to the dynamics and statics of disordered systems.
A. Zegarac and F. Caravelli
https://www.invenia.ca/wp-content/uploads/Square-Business-models-design-space-for-electricity-storage-systems-Case-study-of-the-Netherlands-1.png
Business models design space for electricity storage systems: Case study of the Netherlands
Because of weather uncertainty and dynamics, power generation from some renewable energy technologies is variable. Electricity storage is recognized as a solution to better integrate variable renewable generation into the electricity system. Despite considerable growth in the research on the electricity storage, implementation of electricity storage systems (ESS) is globally negligible because of technical, institutional, and business model challenges. We use literature review and data analysis to provide a conceptual framework and a design space for ESS business models in the case of Dutch electricity sector by taking technological, institutional, and business model considerations into account. We provide a map of single-application business models for ESS in the Netherlands which can be used as a basis for making ESS application portfolios and evaluating ESS business models in other parts of the world as well. Furthermore, this research can be used to inform models that explore the evolution of ESS.
S.A.R. Mir Mohammadi Kooshknow, C.B. Davis
https://www.invenia.ca/wp-content/uploads/Square-The-Gaussian-Process-Autoregressive-Regression-Model-GPAR-1.png
The Gaussian Process Autoregressive Regression Model (GPAR)
Multi-output regression models must exploit dependencies between outputs to maximise predictive performance. The application of Gaussian processes (GPs) to this setting typically yields models that are computationally demanding and have limited representational power. We present the Gaussian Process Autoregressive Regression (GPAR) model, a scalable multi-output GP model that is able to capture nonlinear, possibly input-varying, dependencies between outputs in a simple and tractable way: the product rule is used to decompose the joint distribution over the outputs into a set of conditionals, each of which is modelled by a standard GP. GPAR’s efficacy is demonstrated on a variety of synthetic and real-world problems, outperforming existing GP models and achieving state-of-the-art performance on the tasks with existing benchmarks.
James Requeima, Wessel Bruinsma, Will Tebbutt, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/Square-Learning-Causally-Generated-Stationary-Time-Series.png
Learning Causally-Generated Stationary Time Series
We present the Causal Gaussian Process Convolution Model (CGPCM), a doubly nonparametric model for causal, spectrally complex dynamical phenomena. The CGPCM is a generative model in which white noise is passed through a causal, nonparametric-window moving-average filter, a construction that we show to be equivalent to a Gaussian process with a nonparametric kernel that is biased towards causally-generated signals. We develop enhanced variational inference and learning schemes for the CGPCM and its previous acausal variant, the GPCM (Tobar et al., 2015b), that significantly improve statistical accuracy. These modelling and inferential contributions are demonstrated on a range of synthetic and real-world signals.
Wessel Bruinsma, Richard E. Turner
https://www.invenia.ca/wp-content/uploads/Square-Correlations-and-Clustering-in-Wholesale-Electricity-Markets-thumbnail.jpeg
Correlations and Clustering in Wholesale Electricity Markets
We study the structure of locational marginal prices in day-ahead and real-time wholesale electricity markets. In particular, we consider the case of two North American markets and show that the price correlations contain information on the locational structure of the grid. We study various clustering methods and introduce a type of correlation function based on event synchronization for spiky time series, and another based on string correlations of location names provided by the markets. This allows us to reconstruct aspects of the locational structure of the grid.
Tianyu Cui, Francesco Caravelli, Cozmin Ududec
https://www.invenia.ca/wp-content/uploads/The-mise-01.png
The mise en scène of memristive networks: effective memory, dynamics and learning
We discuss the properties of the dynamics of purely memristive circuits. In particular, we show that the amount of memory in a memristive circuit is constrained by the conservation laws of the circuit, and that the dynamics preserves the symmetry by means of a projection on this subspace. We obtain these results both for current and voltage controlled linear memristors. Moreover, we discuss the symmetries of the dynamics which are due to the circuit cohomology, and study the weak and strong non-linear regimes.
Francesco Caravelli
https://www.invenia.ca/wp-content/uploads/Square-The-complex-dynamics-of-memristive-circuits-analytical-results-and-universal-slow-relaxation.png
The complex dynamics of memristive circuits: analytical results and universal slow relaxation
Networks with memristive elements (resistors with memory) are being explored for a variety of applications ranging from unconventional computing to models of the brain. However, analytical results that highlight the role of the graph connectivity on the memory dynamics are still a few, thus limiting our understanding of these important dynamical systems. In this paper, we derive an exact matrix equation of motion that takes into account all the network constraints of a purely memristive circuit, and we employ it to derive analytical results regarding its relaxation properties. We are able to describe the memory evolution in terms of orthogonal projection operators onto the subspace of fundamental loop space of the underlying circuit. This orthogonal projection explicitly reveals the coupling between the spatial and temporal sectors of the memristive circuits and compactly describes the circuit topology. For the case of disordered graphs, we are able to explain the emergence of a power law relaxation as a superposition of exponential relaxation times with a broad range of scales using random matrices. This power law is also {\it universal}, namely independent of the topology of the underlying graph but dependent only on the density of loops. In the case of circuits subject to alternating voltage instead, we are able to obtain an approximate solution of the dynamics, which is tested against a specific network topology. These result suggest a much richer dynamics of memristive networks than previously considered.
Francesco Caravelli, Fabio Lorenzo Traversa, Massimiliano Di Ventra
https://www.invenia.ca/wp-content/uploads/Square-Trajectories-entropy-in-dynamical-graphs-with-memory.png
Trajectories entropy in dynamical graphs with memory
In this paper we investigate the application of non-local graph entropy to evolving and dynamical graphs. The measure is based upon the notion of Markov diffusion on a graph, and relies on the entropy applied to trajectories originating at a specific node. In particular, we study the model of reinforcement-decay graph dynamics, which leads to scale free graphs. We find that the node entropy characterizes the structure of the network in the two parameter phase-space describing the dynamical evolution of the weighted graph. We then apply an adapted version of the entropy measure to purely memristive circuits. We provide evidence that meanwhile in the case of DC voltage the entropy based on the forward probability is enough to characterize the graph properties, in the case of AC voltage generators one needs to consider both forward and backward based transition probabilities. We provide also evidence that the entropy highlights the self-organizing properties of memristive circuits, which re-organizes itself to satisfy the symmetries of the underlying graph.
Francesco Caravelli
https://www.invenia.ca/wp-content/uploads/2016/08/conformity-driven.png
Conformity Driven Agents Support Ordered Phases in the Spatial Public Goods Game
We investigate the spatial Public Goods Game in the presence of conformity-driven agents on a bi-dimensional lattice with periodic boundary conditions. The present setting usually considers fitness-driven agents, i.e., agents that tend to imitate the strategy of their fittest neighbors. Here, fitness is a general property usually adopted to quantify the extent to which individuals are able to succeed, or at least to survive, in a competitive environment. However, when social systems are considered, the evolution of a population might be affected also by social behaviors as conformity, stubbornness, altruism, and selfishness. Although the term evolution can assume different meanings depending on the considered domain, here it corresponds to the set of processes that lead a system towards an equilibrium or a steady-state. In doing so, we use two types of strategy update rules: fitness-driven and conformity-driven. We map fitness to the agents' payoff so that richer agents are those most imitated by fitness-driven agents, while conformity-driven agents tend to imitate the strategy assumed by the majority of their neighbors. Numerical simulations aim to identify critical phenomena, on varying the amount of the relative density of conformity-driven agents in the population, and to study the nature of related equilibria. Remarkably, we find that conformity fosters ordered phases and may also lead to bistable behaviors.
Marco Alberto Javarone, Alberto Antonioni, Francesco Caravelli
https://www.invenia.ca/wp-content/uploads/Square-Optimal-growth-trajectories-with-finite-carrying-capacity.png
Optimal growth trajectories with finite carrying capacity
We investigate the spatial Public Goods Game in the presence of conformity-driven agents on a bi-dimensional lattice with periodic boundary conditions. The present setting usually considers fitness-driven agents, i.e., agents that tend to imitate the strategy of their fittest neighbors. Here, fitness is a general property usually adopted to quantify the extent to which individuals are able to succeed, or at least to survive, in a competitive environment. However, when social systems are considered, the evolution of a population might be affected also by social behaviors as conformity, stubbornness, altruism, and selfishness. Although the term evolution can assume different meanings depending on the considered domain, here it corresponds to the set of processes that lead a system towards an equilibrium or a steady-state. In doing so, we use two types of strategy update rules: fitness-driven and conformity-driven. We map fitness to the agents' payoff so that richer agents are those most imitated by fitness-driven agents, while conformity-driven agents tend to imitate the strategy assumed by the majority of their neighbors. Numerical simulations aim to identify critical phenomena, on varying the amount of the relative density of conformity-driven agents in the population, and to study the nature of related equilibria. Remarkably, we find that conformity fosters ordered phases and may also lead to bistable behaviors.
Marco Alberto Javarone, Alberto Antonioni, Francesco Caravelli
https://www.invenia.ca/wp-content/uploads/Square-Neurogenesis-Paradoxically-Decreases-Both-Pattern-Separation-and-Memory-Interference-1.jpeg
Neurogenesis Paradoxically Decreases Both Pattern Separation and Memory Interference
The hippocampus has been the focus of memory research for decades. While the functional role of this structure is not fully understood, it is widely recognized as being vital for rapid yet accurate encoding and retrieval of associative memories. Since the discovery of adult hippocampal neurogenesis in the dentate gyrus by Altman and Das in the 1960's, many theories and models have been put forward to explain the functional role it plays in learning and memory. These models postulate different ways in which new neurons are introduced into the dentate gyrus and their functional importance for learning and memory. Few if any previous models have incorporated the unique properties of young adult-born dentate granule cells and the developmental trajectory. In this paper, we propose a novel computational model of the dentate gyrus that incorporates the developmental trajectory of the adult-born dentate granule cells, including changes in synaptic plasticity, connectivity, excitability and lateral inhibition, using a modified version of the Restricted Boltzmann machine. Our results show superior performance on memory reconstruction tasks for both recent and distally learned items, when the unique characteristics of young dentate granule cells are taken into account. Even though the hyperexcitability of the young neurons generates more overlapping neural codes, reducing pattern separation, the unique properties of the young neurons nonetheless contribute to reducing retroactive and proactive interference, at both short and long time scales. The sparse connectivity is particularly important for generating distinct memory traces for highly overlapping patterns that are learned within the same context.
Becker, Finnegan
https://www.invenia.ca/wp-content/uploads/Square-Correlation-Structure-of-Spiky-Financial-Data-The-Case-of-Congestion-in-Day-Ahead-Energy-Markets.png
Correlation Structure of Spiky Financial Data: The Case of Congestion in Day-Ahead Energy Markets
I study the correlation structure and argue that these should be ltered. I propose the use of di erent correlation measures other than Pearson, in particular a modi cation of Event Synchronization adapted to negative values or a ltered correlation matrix.
Caravelli
https://www.invenia.ca/wp-content/uploads/Square-On-Moments-of-the-Integrated-Exponential-Brownian-Motion.png
On Moments of the Integrated Exponential Brownian Motion
We present new exact expressions for a class of moments for the geometric Brownian motion, in terms of determinants, obtained using a recurrence relation and combinatorial arguments for the case of a Ito’s Wiener process. We then apply the obtained exact formulas to computing averages of the solution of the logistic stochastic differential equation via a series expansion, and compare the results to the solution obtained via Monte Carlo.
Caravelli, Mansur, Severini, Sindoni
https://www.invenia.ca/wp-content/uploads/Square-Bounds-on-Transient-Instability-For-Complex-Ecosystems-1.png
Bounds on Transient Instability For Complex Ecosystems
Stability is a desirable property of complex ecosystems. If a community of interacting species is at a stable equilibrium point then it is able to withstand small perturbations without any adverse effect. In ecology, the Jacobian matrix evalufated at an equilibrium point is known as the community matrix, which represents the population dynamics of interacting species. The system’s asymptotic short- and long-term behaviour can be determined from eigenvalues derived from the community matrix. Here we use results from the theory of pseudospectra to describe intermediate, transient dynamics. We show that the transition from stable to unstable dynamics includes a region of transient instability, where the effect of a small perturbation is amplified before ultimately decaying. The shift from stability to transient instability depends on the magnitude of a perturbation, and we show how to determine lower and upper bounds to the maximum amplitude of perturbations. Of five different types of community matrix, we find that amplification is least severe with predatorprey interactions. This analysis is relevant to other systems whose dynamics can be expressed in terms of the Jacobian matrix. Through understanding transient instability, we can learn under what conditions multiple perturbations—multiple external shocks—will irrecoverably break stability.
Caravelli, Staniczenko
https://www.invenia.ca/wp-content/uploads/Square-Multi-scaling-of-wholesale-electricity-prices-1.png
Multi-scaling of wholesale electricity prices
We empirically analyze the most volatile component of the electricity price time series from two North-American wholesale electricity markets. We show that these time series exhibit fluctuations which are not described by a Brownian Motion, as they show multi-scaling, high Hurst exponents and sharp price movements. We use the generalized Hurst exponent (GHE, H(q)) to show that although these time-series have strong cyclical components, the fluctuations exhibit persistent behaviour, i.e., H(q)>0.5. We investigate the effectiveness of the GHE as a predictive tool in a simple linear forecasting model, and study the forecast error as a function of H(q), with q=1 and q=2. Our results suggest that the GHE can be used as prediction tool for these time series when the Hurst exponent is dynamically evaluated on rolling time windows of size ≈50−100 hours. These results are also compared to the case in which the cyclical components have been subtracted from the time series, showing the importance of cyclicality in the prediction power of the Hurst exponent.
Ashtari, Aste, Caravelli, Di Matteo, Requeima, Ududec
https://www.invenia.ca/wp-content/uploads/Square-Scale-free-networks.png
Scale-free networks as an epiphenomenon of memory
Many realistic networks are scale-free, with small characteristic path lengths, high clustering, and power law in their degree distribution. They can be obtained by dynamical networks in which a preferential attachment process takes place. However, this mechanism is nonlocal, in the sense that it requires knowledge of the whole graph in order for the graph to be updated. Instead, if preferential attachment and realistic networks occur in physical systems, these features need to emerge from a local model. In this paper, we propose a local model and show that a possible ingredient (which is often underrated) for obtaining scale-free networks with local rules is memory. Such a model can be realised in solid-state circuits, using non-linear passive elements with memory such as memristors, and thus can be tested experimentally.
Caravelli, Di Ventra, Hamma
x x
Invenia
This is us, drop by sometime.