When you work for a software company that’s a household name making sure your data is secure is a top priority. We recently sat down with Shannon Orr, Assisted General Counsel at Intuit, and discussed how and why legal teams should be protecting what’s most important to their organizations. We also dug into privacy law, why and how Shannon started to focus on this in her career. If you’re working to protect your companies intellectual property this is a great episode to listen in to!
Episode transcript
Read the full transcript
Made with speech-to-text software, so a few words may be off.
0:00Welcome to another episode of Contract Heroes, the show where our guests, and sometimes us, provide best practice recommendations on all things related to contract management. Hi, everyone, and welcome to another episode of Contract Heroes, where we're not the heroes, our guests are. And today on the show, we have, uh, Shannon Orr from Intuit, and Shannon is the assistant general counsel over there. She has a, a large focus on privacy and AI. Uh, before we, uh, I s- turn things over to Pepe to, to dig into a little bit of what we're gonna be talking about today, I just wanted to flip things over to Shannon. Uh, Shannon, would love to hear a little bit more about your, about your background, uh, you know, how you got into, uh, this, this space. I, I think it's pretty unique.
0:58And, um, yeah, so I'll, I'll let you, uh, take over. Hi. Yeah, thank you so much for having me. Um, just as a point of kind of form- like, formality, um, I'm gonna be talking about myself today and not, not Intuit's position. So I, that's, I'm sure most of your guests kind of preface that. But just to be clear, this is about me and my, my personal space and my work in, in privacy and some of the things I'm doing, um, but not a reflection of, of Intuit's positions. Um, so yeah, my background. I, uh, graduated from law school and thought I was gonna go into real estate law because my family's in real estate. And, um, turns out that I graduated at, like, the worst year in the history of practicing law to even consider going into real estate. Um, and so I moved back to Washington, DC, and I was, um, having some conversations with, uh, some lawyers who were kind of leaving big, big law practice and starting their own kind of boutique shops.
1:58And, and I ended up, uh, having a conversation with a woman named Karen J. Newman, and who now is, uh, one of my favorite attorneys and, um, someone I work with closely all the time. And, uh, our first conversation was about, you know, what are all the ways that the government can track you through a toll booth? And thinking through, you know, the use of surveillance equipment, the use of financial tracking, what it means to attach a credit card to your license plate. And, uh, I think we had, like, an hour-long conversation that stemmed into government surveillance, and I realized, you know, I was never going back. Um, and I ended up working with Karen, and then she ended up later becoming, uh, the chief privacy officer of Department of Homeland Security, and I kind of took a different path, um, after working with her to go in-house back in Silicon Valley, and I've worked in-house at a, at a few different, uh, companies here in, in the Bay Area, um, both in the B2B space as well as in the retail space, and now in the consumer-facing and small business space. Um, so, uh, my work has focused primarily on the use of data, the collection of data, how we share data, how we, um, can leverage data to be more strategic.
3:18Um, and then also, you know, how we protect, uh, our customers and how we protect the world at large when we're talking about, um, issues of data collection and sharing and use. Um, and most recently, uh, my practice has turned into focusing on, you know, frontier technologies, including artificial intelligence, um, and, um, new things that are coming out in that space as well. Thanks a lot, Shannon, and we are very excited to have you in the show. And now that, that, that you're mentioning it, we wanted to focus a lot this interview especially on artificial intelligence, right? What does it mean for companies when they try to im- implement those kind of tools in their organization, and some of the privacy law, uh, issues that a company may face when they try to implement those tools, right? Because at the end, we want to automate a lot of the work, and these tools can be very useful for the companies.
4:19But there are particular things that you gotta pay attention in order to avoid any kind of fines or, or, or something that c- can put into a big risk the organization. So before we go deeper into the technical part, uh, for the people that is not involved or is not, uh, related to anything on, on, on privacy, can you talk us a little bit on what is privacy law, and how did you get into it? Ah. Yeah, for sure. So, um, privacy law is the jurisprudence around, uh, around privacy, which can be defined in, in many different ways, but generally is defined as, like, the state or being free of company or observation. And traditionally, um, at least in America, uh, privacy law has kind of been, um, bifurcated into different types of privacy.
5:13So, uh, traditionally, uh, some more classical forms of privacy are, like, territorial privacy, for instance, which is, uh, the right to have freedom of actual physical space. And so a, uh, control in privacy, um, a very kind of, like, non-technical control would be, like, building a fence, for instance, or having a lock on the door, or even having a door at all. That, that provides you some privacy. Um, there's also bodily privacy, so the right to be free of intrusion into your physical body, um, that includes, you know, elements of medical privacy. Also elements of, um, sexual privacy or privacy, um, from assault. Um, and then there's also communications privacy, so the freedom to have communications without observation. Um, that traditionally is looked at, you know, uh, lawyers would look at wiretapping laws, for instance. You know? Right. Is someone able to listen into your conversations?
6:12Now, it's also, you know, can they listen to your, uh, text messages? What can your employer do or what can your school do if they control your device? Um, and then most recently, privacy has kind of taken, um, a turn, um, starting, uh, really i- in the turn of, of this century or last century into informational privacy, which is, um, you know, how, how do we use information about individuals to kind of track them or monitor them, um, observe them? And, uh, what's interesting about technology is that technology allows for the intersectionality of those types of privacy to come together. And so today, you know, when we talk about privacy, we talk about, you know, in how informational privacy may interact with communications privacy or how we may be able to observe someone now in their home, um, by virtue of putting, uh, new types of tracking measures or data measures in the appliances you use or in an electrical gr- in an electrical grid. Um, and most recently, I think there's actually an evolution, and this is really, I think ta- has kind of gotten some speed over the past couple years, which is about the concept of algorithmic privacy. Right.
7:33Which is the idea that people should be either free of or have control over predictions or insights that are made about them based on the intersectionality of those other forms of privacy. So how your data, you know, if we can get a composite view of how you, for instance, uh, spend your money or go about your daily routines or the people you interact with on social media or your types of jobs or how you move in the physical space and combine them with things like perhaps genetic information or communications history, that allows us to make predictions about you, um, based on different types of data science, and we can use those predictions to either, um, help you or hurt you. And so a lot of, a lot of the conversation around privacy today, and also I think the ethics in, in AI, um, is around that concept. Yeah.
8:27And, and, I mean, there's a lot of buzz out there, especially after all these Facebook movies on, on Netflix when they got a lot of data breaches and, uh, and a lot of breaches with the, with their privacy laws. But can you give us just, like, an example or any other, uh, um, story or experience that how a company has to handle that information? Especially, like, compliance works. What do you have to do when you have access to that information? Yes. That is a really important question. So depending on, um, de- depending on what your company does, uh, so if you're a very... If you're in a regulated space, there's probably a chance you already have some obligations that you are aware of. So for instance, if you work in a medical field or if you work in a financial field, um, even in an aca- in, in an academic sense, um, you may be collecting and using information in a way that is already governed.
9:27Um, if you also work internationally, there are different frameworks for compliance that, um, are also very important. So most notably, a lot of people know about the GDPR. Yeah. Um, which is a, a law in, in Europe that is a comprehensive privacy scheme, um, that covers all data. Um, in the United States, we've taken a little bit of a different approach to our privacy jurisprudence. We think about it as, um, kind of contextually or to protect certain types of harm as opposed to protecting all data about a- all people. Um, Europe's taken a different approach. Um, and, uh, the laws can be more stringent, and, uh, regulators in Europe, um, are, are very well-versed in their privacy law. Um, and the European Privacy Law has been super influential, um, a- with the rest of the world.
10:21And so even if you're a United States company operating in the US, you may find, um, that, uh, your employees or that your customers may be more sensitive, um, to all of their data, not certain subsets of data, and you may wanna consider, you know, the implications of, of international law as well. Um, getting back to your question, you know, what should companies consider when thinking about their privacy obligations, I mean, first and foremost, it kind of starts with, uh, you know, what is the expe- regardless of your, of your legal obligation, I always like to ask the question first of, you know, what is your customer's expectation? Okay. And, and start there because I think that the answer to that question is gonna naturally lead you into, um, what you should be doing under the law and also what you should be doing, um, in terms of having a, uh, a socially conscious or ethical position for your company in terms of data.
11:22Um, notif- first steps are, you know, what data are you collecting? What do you actually need? Um, what are you telling people about how, what you're collecting? And then how are you using it? Are you using it in ways that, um, your customers, uh, expect you to be using it? Are you also using it for subseq- like, subsequent uses that may be less expected? Um, are you sharing it with third parties? Are you sharing it with vendors? Are you sharing it with the government? Are you sharing it with partners? Are you sharing it in academic, um, studies? There's a lot of different ways you could be sharing or even selling data, and those types of obligat- those types of activities bring on their own obligations as well. Um, th- And lastly, how are you protecting it when you have it in, when you have it in your environment? Um, are you aware of, uh, the security obligations that you may have to think about for your data? If you have super sensitive data or you work in a regulated field, you may have, um, some concrete security obligations that you need to pay attention to as well. Got it.
12:25So like an example is do, do you have to fill any kind of report on, on what are you doing to that information? Report, so when, when you ask about report, like a report internally or a report externally, externally? Externally, a 2020, uh, govern- governmental office. Oh. Um, so depending on where you do business, you may have, um, some reporting obligations or you may have, uh, transparency obligations. Right. I would say first and foremost, no matter where you're doing business, you m- should be telling, uh, the people whose data you're collecting, you should be giving them some sort of notice, and usually that's in some kind of privacy statement. It may be in a formal notice that you provide to your customers, for instance like a HIPAA notice, um, or something you present to them like when you're actually doing services or providing services or products.
13:25Um, but most companies I think rely on a, on a publicly facing privacy notice. Okay. In... Yeah. Got it. It, it, and Shannon, you know, I mean, how often I, I guess in more, more general terms, I mean, in, in this space, I mean, how often are organizations out there sharing the information they're collecting from their end users with other entities? I mean, is that a normal practice or, or is that something that's kinda more of a one-off? I, I feel like, you know, I, I think on a, on a B2C level, you know, we obviously just kind of scan high level through contracts, you know, and, and just click accept. So I think in those scenarios, I, I think we probably miss stuff. But I mean, in a, like in a B2B space how often is, is this actually happening? Yes. So I, I think very, very frequently. I, I, I don't know how a company can do business today without sharing some form of information with a- another third party. Like, I think that's, that's very rare. Um, 'cause even if you're a very small organization, y- I'm sure you're leveraging technology in some way that requires you to th- at least think through the level of access that that vendor might have, for instance, right?
14:39So I would say and to answer your, your question specifically, I think it's pretty frequent that companies share with, with third parties. And the way that I kind of think through a third party sharing scenario is, you know, do you have different entities within your own family of companies that you're sharing? 'Cause those could be, that could be a sharing with a third party. Mm-hmm. Are you sharing with vendors? Are you sharing with the government? Do you have an obligation to share with the government? And, like, do you understand the parameters of those sharing obligations? Um, are you sharing with partners? For instance, are you, you know, do you have, uh, marketing partners, or do you have a distribution network? Are you sharing with, uh, co-collaborators? Um, are you building products together or building services together? You know, those, those are types of sharings as well. Are you selling data? Um, there are a number of, of organizations that, uh, you know, part of their business model involves selling certain part- certain- Right... parts of data to other companies. Mm-hmm. Yeah.
15:42Um, in those instances there may be additional obligations and protections you need to consider as well. Yeah, and I feel like every time I'm, I'm filling out a, a form to download a white paper or something like that, I'm wondering how soon I'm gonna get a call from another organization, and it seems to have been pretty quick. You know, I, I just changed my phone number, and I've already got salespeople reaching out to me on my new line. So, you know, it's, it's- No stop... it's kind of expected. No, no it's not, you know? And it's, it's, um, you know, it's interesting or, you know, I mean, when, when you're thinking about something and then it pops up on Instagram, it's like, you know, where, where, where are you guys listening from? So it's, it's, um, in, in both the work and, and, you know, at home, it's, it's, uh, interesting to ha- to see and, and get an understanding of, you know, who has this information and, and how they're getting it. Um, but, you know, I, I think, you know, a big part of this conversation, what we really wanted to talk about was, you know, the, the AI portion of, of, um, privacy law and, and how that's evolved over the past couple of years.
16:39I mean, can you just give us kind of a, a high level of, of how those two kind of merged and, and, you know, what maybe the next couple of years hold for th- th- that space and, and privacy? Sure. So let's, let's talk about, um, you know, what is AI? So AI, I mean, even providing a definition of artificial intelligence is, is, uh, a worthy topic- Mm-hmm... because I don't think there's like one working definition in, a- across the globe right now. Um, but at a high level, you know, AI is, uh, developing intelligence through artificial means. And that can include things all the way like robots, you know, talking and acting and feeling. It can also, um, involve things like, uh, automated vehicles and cars and everything from exploring, uh, like natural language processing or emotional intelligence and how we kind of think through those things.
17:39Um, in order for AI at least to function today, there are three things that are necessary. So one, you need to have really advanced computing power. Um, so in order to process, uh, an algorithm and data you need really advanced computing power. You also need an algorithm. An algorithm essentially is like a recipe for telling, uh, the computer how to process certain data. Um, and then you also need a lot of data, a lot of data. Uh, there are, are, there are discussions, um, right now at least in, in, i- in computer science for AI on how we may be able to use less data to have, uh, you know, efficient AI. But right now, in order to train an AI system, you need a lot of data. And privacy, you know, any time that you're, you're generating millions and millions and millions of data points, there's gonna be, I think, natural privacy questions about that, right?
18:40Like, how are you even collecting that information? How are you storing that information? How are you thinking about the individuals whose information is collected in that data set? And then, you know, what's really interesting about AI specifically is what is the results of an AI system? So when you create an insight about someone, when you create a prediction, what, what is the ethical boundaries of that, of that creation? So for instance, if I... There are now, um, algorithms that can help predict your race, that can help predict your gender, that can predict, um, how you may take certain actions. Um, there are notable studies about, um, other technology companies who are playing with the idea of being able to evaluate your emotions through... like, even from facial recognition or, um, by the way you interact with a chat bot. Um, those types of things, um, are deeply, deeply personal insights about individuals.
19:45And one, I think there's always a question of, you know, should we even be creating technology that makes those insights? And then secondarily, if we do create that technology, is it accurate? Do we, do we know really, like, what someone's race is? Like, how do you even define race in a way that could, uh, allow for a true indication of how someone self-identifies, right? And then h- how... what do you do with that information? Like, once you have it, do you keep it to yourself? Do you use it to build new products and services? Do you tell them that you've created that insight? Do you give them a right to that insight, or do you consider that proprietary? Do you, you know, perhaps allow them to correct it or to self-report it? Like, how do, how do we think about those questions? Um, and you know, and how we address that, I think, is, is really where the conversation is today.
20:45Um, in addition to that, you know, AI, because, uh, the more, the more complicated an AI system is, the more difficult it can be to explain why it's made a prediction. And there are certainly a lot of efforts today in computer science to figure out, you know, how to build, uh, explainable AI systems. But at least today, there are certain systems where the calculus becomes so abstracted that we may not know exactly why a prediction was made. And so even if we try to correct certain things in a system, it, it... the data itself may, um, self-replicate those, uh, types of corrections. So for example, if let's say we decided that we thought race was not important in or, um, inappropriate to consider, um, in a particular, uh, decision-making process.
21:50If we remove kind of all of the fields of race and even proxies of race, the data that we're using to make predictions is real world data. It's data that is a reflection of our society today. It's data that is a reflection of how we interact with each other today. And because an algorithm is, is trying to find the most efficient way to make a particular prediction, it may inherently use demographics or historical data that has race kind of interlaced in there anyways. And so there's questions today about how do we even build systems that reflect our values, including values around privacy, um, when we're dealing with real world data? Because we... Right now, unless you change society as a whole, you may not be able to change the results of an AI system. And what is the best way or how is the best way to let to know to your clients that you are gathering that information, let's say for the research and development, the department in, in order to, uh, make your software better?
23:00Do you have to sign any kind of ag- of agreement, or just showing that in the privacy, uh, notice or, or policies, it's that enough? It may depend on the type of technology you're developing and also the type of data. So, uh, for example, um, as a general rule of thumb, uh, at least in the United States, you don't, you don't necessarily need to get consent or p- um, in order to, to use personal information for development of new products and services. However, if you're collecting sensitive demographic information, so for instance, that would, could be your race or your gender or your sexual preferences, medical history, um, employment history, th- like, if, records of whether or not you've had a criminal conviction or if you're a part of a trade union, um, your political beliefs, your religious beliefs, those types of demographic information, you may have an obligation to ask formally for consent. Um, also, if you're developing certain types of technology, for example, or y- or even using certain types of technology, like biometric technology, so, uh, things that could scan your irises or use facial recognition- Um, or even a voice imprint, uh, or fingerprints even.
24:14Um, if you're using those types of technologies there are now certain laws that require you to provide, um, certain advanced, uh, notice, also get consent, and also have certain rights to how to opt out or delete that information. Great. Okay. And, and Shannon, you know, uh, for, f- right, you've been at some, some fairly large companies and, and it seems like you've obviously b- you know, been focused on this a lot. I mean, what, what types of advice would you give for, um, you know, smaller legal teams who, who don't have somebody that's, you know, specifically focused on security when they're, they're trying to negotiate with, with other organizations and they're reviewing, you know, security language inside of these, these contracts that they're dealing with? I mean, what, what types of tips could you give folks to make sure that they're, they're covering their, their organization and, and also trying to, to be as, I guess, polite as possible when, when negotiating with other, you know, legal teams that are out- outside of their company? Sure. That's a great question.
25:17I love your point about being polite. 'Cause I think as lawyers we can get super feisty, right? Um, I think that's always a good reminder, to be gentle with all the people we're talking to. Um, you know, I think that AI in particular is going to do wonders if, because it's going to be able to allow smaller organizations to leverage, um, big data and, and data science in a way that they otherwise couldn't build for themselves. And these tools are gonna be easier and easier to, um, leverage in your back-end environments, like, for, you know, to, to run your business. Um, but also to, to benefit your customers, whoever they may be. And, um, there's some, there's a few things that, that should be considered. And so, one, I would always reflect on, you know, if you're going to really use any, any product, but especially a third party's AI, you know, consider, you know, what information do they need from you?
26:26Are they, um, are they just giving you an algorithm and you're gonna run it in your environment and it just stays in your environment? That might be a pretty easy use case. But if their algorithm is also, you know, leveraging your data and going to take learnings from your data and apply it elsewhere, you know, that's something that you should consider, particularly if you have sensitive information or if you think your customers really wouldn't like that. Uh, and a way to kind of look at that in a contract is to kind of look at product improvement language. So, you know, how, how is an algorithm trained? How... Like, does the third party want to, um, have an ability to improve its products and services through the use of your data? I think that's a really important, um, thing to consider. And now, you know, there are some AI companies, um, some of the bigger ones, especially in more regulated spaces like in the medical field, for instance, um, that have said, "Hey, you know what?
27:27For these types of products and services, we're gonna make sure the algorithm isn't trained on your data. So we, we have already developed a special way to train and improve this algorithm for your benefit without compromising the privacy of your customers or your employees." I think that's a really important thing to consider. Um, the other thing to consider is, you know, how could this algorithm be used in a different context? So I think a really, um, really interesting use case, uh, uh, is facial recognition software. Now, there's a, there's a couple of companies that have come out, um, with various forms of facial recognition. And, um, this facial recognition software, the algorithm can be leveraged in, in many different ways. So it can be used, uh, in, to clock employees in, um, to a product and service. It could be used, um, to, uh, make sure that, uh, the people who are handling your customer data are actually the people that you've approved. I mean, there's a lot of positive ways that you can use that algorithm.
28:36That algorithm can also be used in large scale surveillance equipment. So to monitor mass amount of people, um, or it can be used by law enforcement to monitor and, um, to monitor, uh, possible criminal suspects. And so it's important to kind of consider apart from your contractual arrangement, you know, how is this company leveraging this? Because the, outside of your legal obligation, I think the, the PR issues are really, really important in this space. Um, because, you know, by leveraging technology with your customers, that could also maybe jeopardize other parts of their lives, um, is something that's really important to consider. 'Cause right now, you know, the law is, is just starting to consider all of these, these aspects, and technology's gonna move much faster than the law is, is going to catch up, right?
29:34And so considering the possible PR implications, um, I think is really, really important. Yeah. Yeah, I, I, I, you know, I think those are some great takeaways, especially as, as more and more organizations, especially the smaller ones, really start to, uh, invest in technologies that h- that use AI. Um, you know, it's becoming more cost-effective. People are starting to really wrap their head around, uh, different ways that it can be used inside organizations. Uh, although I think a lot of people have different ideas of what AI is actually capable of, um- So I, I think that hopefully over the next couple of years, you know, that'll evolve and, and folks will start to narrow their focus on, you know, really what is AI and, and what more so is just, uh, software getting better at, at being software. Um, but you know, typically we ask a, a couple takeaways at the end of, uh, these calls, but I, I think those last ones you just mentioned really kind of wrap things up.
30:30So I'm gonna, I'm gonna leave you with this one instead. What, what is your, your favorite movie, uh, that, that involves AI? Ooh. Oh, such a good question. You know which one I really like? Is Ex Machina. Okay. Have you seen it? I watched it, but it w- It's, it's been a while. It's been a while. It's fantastic. It's really good. And, uh, I don't wanna spoil it for anyone, but there's some excellent twists. Um, but that deals with artificial general intelligence- Mm-hmm... which is like the idea that, uh, we can create a robot or a body that, you know, acts as a, acts as a human does. Um, and so it's a it's a fun, it's a fun space. It's a little bit, um, it's a little bit scary, which I think everyone should be a little bit scared of AI, at least right now. Yep. Um, and so it... Yeah, it's fun. That's a good question. Well, good. Good. Yeah. Hopefully, uh, Pepe, maybe we should start mixing those in a little bit more at the end of, uh, end of our calls. That was, I liked that one.
31:26That's a good one. Yeah. Very different. Hey, Shannon. Well, well, thanks so much for joining us today. And, uh, if folks wanna connect with you or learn more about, uh, you know, your role and, and, uh, just information in general on, on privacy or AI, where, where th- should they go to find you? LinkedIn. Yeah. Meet me on LinkedIn. Um, I'm Shannon McKenzie Orr, um, and you can find me there. All right. Well, thank you so much. Great. Yeah. Bye. Hey, thanks everybody, and, uh, thanks for giving this a listen. Shannon, thanks so much for joining us, and we hope to, uh, have you back here for another listen on another episode of Contract Heroes. And have a great day.