Future Represented

     

My thoughts on risk from a decade of expertise (also, where have I been the last 6 months?)

April 7, 2025

My experience with AI & risk

I have been thinking about and researching risk, though maybe not by that name, since the end of my bachelor's in ecology, around 2011. My original master's thesis project was to use "pattern recognition techniques" on phyolgenetic trees and functional trait databases to understand (recovery from) catastrophes and mass-extinctions at a community ecology level. When I read about representation learning though, I got sucked in, did my PhD at Mila on generalization, watched it become the world's largest academic AI lab, became a professor at Toronto, and as I saw AI begin to take over silicon valley and then the rest of the world, made responsible AI my primary research area, and last fall rejoined Mila as a core professor.

It's mostly in the last 5 years or so that I've been seriously researching risk again -- mostly in the context of risk assessments involving AI, but also modelling methodologies that use AI to better forecast and understand risks -- coming back to my original research intentions, but with a view to the future more than the past. In 2020, these methods helped form the core of Proactive Contact Tracing, which was almost used as the Canadian government's official app (they went with one based on basic binary contact tracing, developed by Shopify, one of Canada's most successful tech companies, instead). In 2021 I was a fellow at the Cambridge Centre for the Study of Existential Risks. At workshops and symposia on new areas for AI research, AI safety, ethics, sociotechnical alignment, etc., I've given presentations about ecological risk, risk assessments, extreme weather events, sustainability, and resilience.

I think there are issues with the terminology and history of "risk", but I made a pretty cynical decision to frame a lot of my research around that word because of the practical hooks we have into "risk" as a thing we want to do something about, as individuals and societies. All the early international governance documents on AI (the EU Commission's 2019 Ethics Guidelines for Trustworthy AI, China's 2019 National Ethics Subcommittee on AI Report on AI, the 2022 US AI Bill of Rights, et al.) mention risk, avoiding risks, performing risk assessments. Who was studying AI risk and risk assessment in 2019? Fewer than half a dozen folks explicitly, and I correspond with most of them still. Certainly AI safety researchers, ethicists, insurance companies, policy folks, lawyers, and many others were thinking about it one way or another. But despite it being an ostensibly important foundation of AI regulation and global safety, there was (is) no established, holistic scientific field devoted to understanding AI risk.

Risk in my life

I've been having intermittent chronic pain issues for about 7 years. I saw doctors, physiotherapists, and ergonomic specialists; I did lots of stretches and yoga and did all kinds of elimination diets for allergens, and consistently took the iron -- the only thing all the doctors seemed to agree about; I was clearly anemic. There were good days and bad days, and I chased correlations and products and service dog training and I eventually got a set of practices going where I could do about 4 hours of typing/clicking a day without debilitating levels of pain - more than that and I would be woken up feeling like my arms' ligaments were being scraped and zapped by tiny evil machines. I did less yoga, because long stretches of mindfulness made me more aware of everything hurting - it was easier to live my life by tuning that out.

And I was having a great life! I cannot imagine a job I'd love more than being a professor, and now I get to do it in the city that feels like home to me. What a great time to be working on Responsible AI, I thought to myself -- I'd been doing some international travel, and it seemed like every nation in the world was watching the EU AI Act with approval, listening to scientists and researchers who were talking about real-world harms and safety and existential threats and nodding thoughtfully. My own work on deep risk assessment was getting some traction, and I started a collaboration with lawyers, drafting risk assessment protocols that I really believe could cause tangible reductions in harms and risks. In rooms painstakingly chosen by third parties like the UK and Singapore and Canada, the US and China were even nodding thoughtfully in each others' directions. The Paris Action Summit was coming, and I was cautiously optimistic about it being a keystone of international collaboration on AI - an opportunity to get global consensus that this technology needs to be developed carefully, responsibly, allowing time for science to catch up to business, for the good of all above any individual/company/country's profit, recognizing that races to the bottom of safety or literal AI-arms races were in no ones' interest.

So I was busy, you see, and found it relatively easy to ignore the increasingly increasing pain through the fall and winter. And the week before leaving for Paris was not at all a good time for the chronic pain to turn into sharp lower-back attacks that winded me at random times, sometimes almost made me pass out. I saw a doctor, they did an ultrasound. I was so incapacitated for the next few days that my practices fell apart, I only got out of bed to walk my sweet dog, who sniffed me and licked me and fussed over me like a worried hen -- and suddenly I felt better one day. What was different? I had a flash of everything I'd been doing for the last 7 years. I resumed all my practices, except I stopped taking the iron. The sharp pains went away, completely. That was great news, because I was teaching and then flying to Paris the next day for a pre-Summit event bringing together civil society, researchers, government thinktanks, policy makers, etc. I got a call from the doctor's office just after going through security -- "could I come in today? The doctor wanted urgently to speak with me. No, I will be on an airplane. Okay she will call you in two days then. Two days? I thought it was urgent? Can you tell me anything about what this is about? No, sorry." I watched Magic Mike XXL and a lot of nature documentaries, and tried to sleep. I arrived in the morning and went straight to the event.

And what a nice event it was! I saw old friends and made new ones, and had thought-provoking conversations. The prevailing attitude from the government folks was "we're so glad to be talking with you researchers and civil society folks, you know we mostly hear from companies that this AI thing is pretty great and they've solved the bias and things like that, so this is all a bit sobering, but you know we are really hearing you, what important issues these are!" Indeed. I was particularly happy to see safety and ethics communities, of late and so often acrimoniously divided, uniting behind sustainability (broadly and thoughtfully construed) as a guiding principle. The second day happened to be my birthday, and we were planning a dinner and fun evening. As the event closed, I was looking to round up folks when my phone rang - the doctor's office. I went into a cloakroom to hear, over the sound of a thousand well-heeled feet eagerly rushing their sobered owners to somewhere they could become less sober, "I have the ultrasound results here. You have an 11cm mass." Many more details, prescriptions for an MRI and blood testing exceptionally sent by email (getting the appointment in the first place had required fax!), so that I could schedule them immediately upon my return.

The surreality continued

Before returning, I joined the Hawaii pod of the Abundant Intelligences project, an indigenous-led ground-up project to reimagine AI in abundant forms. Our pod's first project is Abundant Soils, looking at relationships with the living land, imagining AI systems that could foster rather than kill those relationships, and more prosaically (but excitingly!) integrating traditional ecological knowledge into soil and therefore agriculture and forestry modelling, improving resilience to climate change. I met incredible people doing amazing work, all of it a bingo card for the US attacks on science. The AI summit turned out to be a shitshow; each shiny corporate photo-op (with accompanying social media campaign) outdoing the next in technosolutionist absurdity underscored by eye-popping financial investments, at a time of post-covid inflation and housing crises where most normal people were struggling to afford rent and groceries.I got the second-worst fever I've had in my entire life; for four days I didn't leave the hotel room and my colleagues brought me coconut water. After two days of being freezing cold wearing my wool coat under the covers in the Hawaiian heat, I started sweating, and then got hotter, and more in pain, and then started *vibrating* - I was afraid I was going to have a seizure or something.

There is no US equivalent of 811, where you can talk to a nurse at any time. The last time I went to a hospital in the US (during an internship, at the onset of the chronic pain!) they charged me $4500, and prescribed what I now know to be an over-the-top regime of NSAIDs, which gave me a rebound inflammatory condition that means if I take NSAIDs for pain now, they work great for 6-12h, but then it comes back twice as bad afterward. I could not, did not, string together a plan, I paced around the room panting and crying and vibrating for half an hour. It passed, I slept, and the next day the fever broke. I walked to the beach, and to a botanic garden. I slept. I went to an anti-fascism protest at the State Capitol, and tried to talk to the only counter-protest group - the leader, carrying a Trump banner, wearing dark glasses and shouting "fuck your feelings!" over and over, was followed by a bent-over person with intellectual disabilities who made eye contact with everyone and pointed at the leader and said "he's right! he's right". I asked why my feelings were an important political issue for him, and the leader said "your mother is a whore, fuck your mother", I laughed and asked if he was trying to hurt my feelings? Why? He walked away cackling that he loved owning these libs, loves shutting them down, and the bent-over person said "are you hurting his feelings? I saw you be mean to him! You're being mean! You're an asshole! He's right!"

Living with risk

It would have taken >6months to get the MRI the doctor said I needed urgently; I paid over $1000 to do it privately. US-based colleagues had hundreds of millions in research funds frozen; money that is the livelihood of hundreds of people they are personally responsible for. Students and immigrants with marijuana convictions were being rounded up at their homes by ICE and sent to Guantanamo. In the waiting room, they gave me a waiver to sign with checkboxes about different risks of having an MRI. It said the contrast dye, needed for my situation, carried a risk of 1/100,000 of death. This seemed wildly high to me, and I asked where it came from - they said I didn't need to sign it yet, they would check and let me know. After undressing and putting on a hospital gown the nurse came to answer, and she said "yeah, it's just statistics". Waiting in the not-heated cubicle for the radiology tech, I tried to hold up all my winter clothes (there were no hooks) while searching for literature on contrast dyes, and texted my brother, who is an emergency room doctor on shift, not expecting to reach him. He called me back immediately, and said that's a completely made-up number, there is some controversy about this but basically there is only sketchy evidence contrast dye is ever fatal, at all, and he would take an MRI with contrast dye in a heartbeat. The MRI tech says something similar, he's been doing this for 20 years and no one has ever died. He is very nice, and patiently waits without pressuring me as we talk myself into doing it. He and the nurses and the booking website were all very concerned and considerate about the possibility of me being claustrophobic. I am not. I love small tight spaces. As it turns out, I quite enjoyed the experience and sounds of being MRId, of lying still for 45 minutes knowing exactly what to expect.

There was too much liquid in the mass to be sure it's not cancer; more blood tests were needed (I hate having blood taken). I wrote in my journal: "the uncertainty IS the risk". My responsible AI class was going really well! An incredible group of engaged young scholars - I think it helped that we were able to talk about the news; it made the (already pretty new) course content the opposite of stale, everything was almost too fresh, flashingly bright and shocking.

I am lucky to have wonderful wonderful colleagues and friends and family supporting me through it all. Some of them suggested I should ask chagepetto et al. for advice or summarization, citing situations of many variables and great uncertainty and confusion that they had unravelled with conversational help. I have never, will never, do this. I read a lot, every day, and despite making little to no conscious effort to internalize or remember any of it, somehow my brain sifts and turns all that input into thoughts and takes that are sensible, coherent, sometimes even insightful, and for any given piece I can recall what it's informed by. This feels to be, as near as I can introspect, an important kernel of my human soul, and the idea of infecting it with questionable unsourceable amalgamated-stolen-internet vomit studded with invisible hallucinations and subtle bias is far more terrifying to me than the prospect of having cancer - two risks with which I am now intimately acquainted.

Learning about the risk

But I (most probably) don't have cancer! I have endometriosis. What's that? It's a lot like cancer, in that it's some kind of tissue (in this case endometrial, which is the tissue that monthly prepares a blood-meal for a fertilized egg and that, should no such thing implant, sheds, which is called menstruation) replicating more than it should in a place it shouldn't be -- typically, as in my case, somewhere pretty near the uterus. But it's unlike cancer in that it's pretty much never fast-fatal (endometrial cancer does exist, and is not the same thing as endometriosis). Endometriosis is usually classified as a chronic pain disorder, sometimes an immune or autoimmune condition -- basically a lot the cancer stuff, but not cancer. I've had covid, with long covid, 4 times, and gotten bad food poisoning (when few others or no one else was affected) a dozen times in the last 7 years. If you've never heard of it, no worries, most people haven't, but it affects 1/5-1/10 people with a uterus (and disproportionately, women of colour). That's a lot! What causes it? We don't know. We have very little information about it -- if a white man can't experience it, is it even real? How would you even go about studying that? /s. It took me almost exactly the average, 7 years, to be diagnosed -- 7 years of confusion and uncertainty and convincing myself it's probably nothing, maybe I'm just stressed. I am one of the least stressed people I know.

What's the treatment for it? It used to be almost immediate surgery, and this is what the doctors and gynecologist I had seen had prepared me to expect. But especially for larger masses like mine, surgery carries more risks and potential side effects of its own. These days, like for many cancers, hormone therapies are the first line of treatment. They have side effects, but these usually go away (sometimes sharply) after about 4 months. Sometimes earlier. Around this time also many patients start to experience relief from the pain and immune symptoms. Sometimes the hormone therapy shrinks the mass so much *it disappears*. I found all this out from the endometriosis specialist I saw, who asked me many many questions about the pain I experienced, and listened to me carefully, and noted and integrated that information before coming up with the treatment plan. It was the first time that I had been asked for more detail than rating the pain on a 10-pt scale, and the first time my descriptions were treated as clinically relevant information about which I was a reliable source. Her gynecological exam was the second time in my life lube was used on the speculum. I saw her because I asked the gynecologist if I could be referred to a specialist for treatment. I asked for that because I looked up "questions to ask gynecologist endometriosis" and endometriosis.ca had a list that included asking to be referred to a specialist. I finish writing a longer version of what is now a 2yr old workshop paper on deep risk assessment. I paint. I interview student candidates for fall 2025, and I'm excited about data sovereignty and research-creation and guarantees on sim2real generalization. I unpack boxes in my new home. I try to focus on feeling validated, on knowing, now, on the uncertainty being lower.

What if my brother hadn't reassured me about the MRI? If I hadn't asked to see a specialist? Would chatbots have recommended asking that question? In addition to the confusion and uncertainty and fomented distrust of myself, the contingency of these heavy outcomes, how differently worse someone else's, some other me's, trajectory could have been also feels like risk to me. It feels like a sick pit in my stomach with a residue of shock, like almost getting hit by a car.

Understanding risk

I've been prescribed a medication, and its website recommends to take it on the first day of your period. My period is 1 day late, which is weird because it's very regular. I wait on starting the medication. three days later, feeling like a teenager, I do a pregnancy test, although there is no way ... "can you get pregnant from a hot tub? no honey". But I do the test anyway, and it's negative, and I do another two days later and it also is. The risk of pregnancy is therefore very low, the packaging of the test informs me. Was it zero? My period finally starts, and I start the hormone therapy -- a progestin, which from the pamphlet and other info I understand is approximately like getting your period every day. I guess that this tells the mass (some peoples' masses) to shed, although any and all literature about the medication is at pains to emphasize we do not understand its mechanism of action; we just gave it to a bunch of people and it worked for a statistically significant number of them. The other main one I'll take if this one's side effects are too bad is approximately like getting menopause. There are many variations on these two, and other options too, and why which one works for which person is also not known.

One thing that does seem to be well-known is that iron supplements aggravate, maybe even cause, endometriosis. I realize, slowly and all at once, as I have a hundred times before but somehow it's different this time, that this is sexism, that I experience sexism because I'm a woman, and it doesn't mean any one person is particularly upset or treating me different or anything for being a girl (although that sure happens too), it's systemic and it affects me and there's sort of nothing I can do about it, although of course I'm doing so much just by being who I am and doing what I'm doing, yada yada, but this is what DEI programs are addressing. Or trying to address, or were addressing. Will this blog post be removed from search results in the US for mentioning DEI? I can't be sure, and neither can you. I acknowledge more of it this time, even though it hurts. I feel like I should be angry, or resentful, but I just feel drained, and kind of dirty.

Every day, I feel like the worst day of my period - aching, exhausted, deeply sad, with occasional crippling cramps. I try to see friends regularly. I cuddle and walk every day with my beloved dog, who performs her trained service of licking and pressure to mitigate my chronic pain symptoms, and her untrained service of adorable joy. I wake up, confused and feeling like I've just done something wrong, and that feeling stays all day long. Intrusive thoughts are mumblingly regular, every few minutes or seconds I take a deep breath and nod and notice the thought and know that it's a side effect and watch it go by and try to remember what I was doing. Can I do this for four months?

The end of risk?

And then, after two weeks of that, one day, I feel better -- so good that I feel ready to become superstitious about every insignificant detail of the day. It feels like the moldy carpet full of staples I'd been zombified walking on has been pulled from under my feet and I've only now noticed it in its absence. I don't feel high. I don't feel abnormal; I don't feel unaware or unaffected by scary awful facts like the largest world nuclear power being taken over by technofascists or their inhuman persecution of trans folks and immigrants and women and their dismantling of scientific and journalistic checks on their consolidation of wealth and power, or the Earth being consumed at unsustainable, untenably rapaciously stupid morally bankrupt levels. But, I feel like I haven't felt in 7 years. I feel infinite again after feeling finite for so long I thought that it was my natural maybe even foreordained state. I feel able again, and now a week after that I still feel that way. I no longer feel at risk.

~*~

So what is risk, and how should we deal with it?

~*~

In conclusion: I'm feeling a lot better. I don't feel sure everything is going to be okay, far from it, but I feel safe, and comfortable, and able to deal. I recognize this is a position of huge privilege and responsibility that I am grateful to be in. I want to help others, our societies, our global ecosystem, to feel the same way; I want everyones' risk to be reduced.

I've thought about it a lot, and done a lot of research, and while I think there are a lot of things to be done at a lot of different levels that can make a lot of difference and do a lot of good (risk assessments, audits, harm-reduction, place-based and community initiatives) ... without a shared fundamental intuition about the nature of risk, even with the best of intentions, we are as likely as not to cause more harm to ourselves, others, and the planet. So this is my contribution to that sharing: after all this time, I find that risk consists in uncertainty, confusion, contingency, and distrust.

Intuitively: Risks are things that feel like pain.

If we want to address risks, we need to notice and acknowledge pain, the ways it circumscribes and curtails our certainty, clarity, resilience, and trust. Do not accept being in and ignoring pain, do not accept closing off any part of you for any reason, and don't accept anyone doing it to anyone else. Do not settle for a finite experience of being. Don't let the bastards grind you down.


Utopia of the Planes

October 8, 2024

Last year I began writing a short story, which is in the universe of a sci-fi quadrilogy I've been writing for a decade. I finished it in the spring and distributed it as a zine at FAccT in Rio (where the first copies were printed!) and MUTEK in Montreal. Today, I did a reading from it at the Mila Talent show, and finally make it available online under a non-commercial CC license.

Here is the screen-readable pdf,
the booklet-format pdf (in case you wanted to print it out, fold in half, and staple it, i.e. zine/book format).
and the epub and mobi (thanks Calibre).

I also recorded a full take of me reading the book, if you're into audiobooks or want to feel like you were at the talent show you might like it.
Here is the m4a.

I will be at MAIS with some physical copies; find me if you would like one :).


Coming home to Montreal

September 15, 2024

Newest news! After a wonderful 3 years at University of Toronto, I'm moving to Montreal to become a core member of Mila, with a tenure-track position at HEC. Tiohtià:ke/Montréal is on unceded Indigenous lands. I recognize the Kanien’kehá:ka Nation as the custodians of the lands and waters in this place I call home. To them I offer my deep thanks, my joy at the beauty and diversity here, and my committment to truth and reconciliation (of which this statement is one small step).

"Mila, awesome, congrats! Uh, isn't HEC like, a business school? Do you..." trails off anyone who knows me in confusion. Do I business? I can't say that I do, but it's one of the best management schools in the world. I had something of an epiphany about the importance of human resources and management science last year, thanks to the scholars at Makerere AI lab I had the fortune to visit and discuss AI deployment with. And, I'm increasingly jaded about the impact Responsible-/ FATE-/ Safe-/ Climate-/ Etc-AI research can have from within the AI community -- we have largely handed over decision-making power w.r.t. AI to business management, so I'm excited about working with other experts at HEC to develop a new program in Responsible AI Management (I'm also excited about ways to transfer power and agency back to communities - reach out if you'd like to talk about that!). Every new generation of students I've been lucky to work with has an increased awareness of the socio-environmental impacts of AI/Business, and our responsibility to each other and to the planet to do better. I see a lot of appetite for a program like this, and I'm excited for graduates of this program to be the leaders of tomorrow.

Of course that's at least a few-year project; in the meantime I'm still primarily running a research lab (see my Research Interests below), which finally has a name and website! ERRATA*™ Lab: Ecological Risk & Responsible AI; Theory & Applications *learning from mistakes. I'm not jazzed about the whole idea of branding, but I *am* jazzed about clearer communication, and I think this achieves that - summarizing my research interests, making it obvious the lab is an entity much larger than me, talking up my awesome students/collaborators and the things we do together. I am recruiting for Fall 2025! see lab website for details/advice/instructions.

I'm restarting my open mentorship hours -- these are flexible/by appointment, with priority given to those currently minoritized in ML. BIPOC, queer, global south, disabled, first-gen scholars, category-averse lovely weirdos, you're all warmly welcomed!

If you have contacted me in the last year or so and did not get a reply/autoresponse, I'm sorry! I had long covid and some other health/personal turmoil, and took a mental health break from all social media (highly recommend, not going back any time soon). Also outlook marks a LOT of valid emails as spam. Please reach out again, using my Mila address :).


A dream come true - I'm a professor!

June 5, 2021

New news! I've accepted a tenure-track position in the Faculty of Information at the University of Toronto, where I'll be an affiliate of both the Schwartz-Reisman and Vector institutes. In the near term, I’ll be recruiting students/RAs to work on deep representation learning & predictive methods in ecological modeling and environmental risk assessment, as well as real-world generalization, learning theory, and practical auditing tools (e.g. unit tests, sandboxes). If you’re interested in those positions, interested in collaborating or chatting about those topics, or know someone who is, please get in touch!



Search

I plan to have a more sophisticated search once I have enough content here that it's needed. For now, it's all here; just press Ctrl+F (Cmd+F on mac).


About

I have a lot of things to say; I plan to say them roughly weekly. For more about me, see my academic homepage. Some of the posts will be more personal e.g. significant life updates, but most will be more like short essays. Occasionally they will be short fiction. I'll also aim to do about 1 per month on clusters of papers I've written and/or find interesting. Here's an incomplete list of topics I'm planning to write about, in very approximate order: (let me know if you would be particularly excited about one of them!)


This blog is publshed under a Creative Commons CC BY-NC-SA license. This license enables reusers to distribute, remix, adapt, and build upon the material in any medium or format for noncommercial purposes only, and only so long as attribution is given to the creator. If you remix, adapt, or build upon the material, you must license the modified material under identical terms. You may not, in general use any text of this blog to train or prompt a language model or similar AI system, because they are commercial products, whose outputs are not licensed BY-NC-SA, and you cannot guarantee they or their derivitives will attribute credit correctly or at all. If you develop an AI system that respects BY-NC-SA, go ahead and use my work for it! Also, please let me know about your research :).