How AI is Invading NYC Schools

Resources

More on the spying powers of GoGuardian

More on the creepy hall pass technology SmartPass

Transcript

Yelani: It’s puzzling that we’re seeing our teachers have to use AI and have to rely on it, when they have the certifications and they went to school to be able to teach us these skills and this knowledge. So, yeah, I feel like it’s just being pushed out so fast without any type of pause. Where do we pause? Where do we stop?

Simon: This is Rights This Way, a podcast from the New York Civil Liberties Union, the ACLU of New York State.

I’m Simon McCormack, editorial manager at the NYCLU, and your host for this podcast, your best listen for the civil rights and liberties issues that impact New Yorkers most.

AI is everywhere, including in our schools. But what does the rapid deployment of AI mean for the 900,000 students in New York City’s public schools? What are the guardrails in place for its use? What are the dangers of AI? What are the everyday changes taking place because of its implementation?

Earlier this year, the New York City Department of Education rolled out guidance for the use of AI. The backlash from students, parents, and pretty much everyone was so severe that, in June, the DOE postponed their planned rollout of an AI playbook, which would’ve been a more comprehensive version of the guidance. This postponement was a win for the movement to rein in AI use in schools, and it’s a clear example of the high stakes around this contentious issue.

On today’s episode, we’re joined by students from the NYCLU’s Teen Activist Project who have firsthand experience with how AI is being used in their classrooms. We ask them about how AI is used in their schools, their concerns about the technology, what needs to change about the use of AI in schools, and a lot more.

And now, I’m joined by two guests: Yelani Joseph and Ariana Misha are both New York City Public High School students and members of the NYCLU’s Teen Activist Project. Yelani, Ariana, welcome to Rights This Way.

Ariana: Thank you.

Yelani: Thank you.

Simon: It’s really great to have you both. Very excited to be able to talk to you today. And I want to start off just with a very general question, which is just, what are your feelings broadly about the use of AI in schools, both teachers, administrators using it, and also students using it? And Yelani, I’ll kick it off with you.

Yelani: Sure. So, I feel as though the use of AI in schools is a little peculiar. I definitely feel like seeing it being more so normalized in our schools is definitely something interesting, because we’re seeing it in our lesson plans, we’re seeing students use it to review their work.

We’re seeing it used in a multitude of ways, and I think we’re not seeing as much regulation as we’re seeing as much students actually using it. So, I feel it’s really interesting to see the growth of it over the years, because it’s been slow, but then it’s also been rampant, especially in 2026.

We’ve seen a lot of new laws, new policies, but we’ve also seen a lot of pushback on AI. So, I feel like there’s two sides to it, and it’s really interesting to see.

Ariana: For me also, adding onto Yelani, I remember when… a year ago when it was embarrassing for students to use AI. And whenever in class I used to see somebody using AI, that used to be so embarrassing. It’s just weird that they don’t know the stuff, so they’re just AI-ing it.

But when I heard about the fact that the education board is planning to bring AI in the classroom, and students are going to be using AI as a part of their education, that was really very sad for me. And I was like, “Okay, so all those students were right. Using AI is okay in classroom.” And it was just really weird.

I think peculiar is a really good word for this, that how something that once used to be embarrassing or weird happened to be part of the… or somebody planned that to be part of the educational system itself.

Simon: Yeah. So, that guidance you’re referring to, you feel like maybe it either implicitly or explicitly made it okay to use AI, that AI is coming, it’s okay for students to use, it’s not embarrassing. Is that fair to say?

Ariana: Yes.

Simon: Yeah. Interesting. Okay. Well, so to talk about that for a second, the guidance that you’re referring to on AI in schools that the Department of Education put out, you both are aware of that through the Teen Activist Project, you’ve gone through it line by line, but a lot of students have no idea that it exists.

So, the AI has been implemented, as you’re saying, by students, teachers, administrators, it’s kind of funneling into your schools, but the lack of guidance there, or at least the awareness of the guidance, is I think something to note.

But I am curious… Ariana, you kind of touched on this in your last answer, but I’m curious what your thoughts are on the guidance itself? What are your impressions from it?

Ariana: I remember the very first day I heard about this policy. It was supposed to be sent to every single student and their parents, as they told us. But I checked my dad’s email, I checked my email, my school email. There was nothing like that. I don’t know why.

And I remember the first day I heard about this was from Charlotte, who is part of the NYCLU. And Charlotte told us about this as we were all organizers in the conference room. And I’m not even joking, at that moment, I was having goosebumps. Like, “These kind of AI policies are going to be part of the educational system from now on?” Or, “Something that used to be so embarrassing or weird, now that’s going to be part of the educational policy?”

And then I was trying to feel like, “Okay, why was this happening? What was the purpose of this?” And when I found out that they’re doing this to make ourselves more cope up with the technology or make us advanced, I really felt sad, because that’s not the best possible way. How about making more better textbooks, or how about increasing funding for the high schools, instead of bringing AI technologies in high schools like this?

So, I was really very sad when it was the first time I heard this from Charlotte.

Yelani: Definitely, I agree. I feel as though a lot of students aren’t aware of how AI is being put into the schools. They just know that AI is there and it’s something that’s available to them now.

I know that a lot of my peers that use AI, or have seen AI in their curriculum in the past, it’s not something they immediately noticed. It’s not something that immediately had guardrails on it. It was kind of just being used. For example, I noticed in my own health class, we noticed that some of the notes that were being put on our slideshows were completely AI generated, and to the point where the words were just gibberish, for lack of a better term. And we were genuinely confused. We were like, “Wow, our curriculum and our notes and the stuff that we’re supposed to be relying on for our core classes like ELA, math, science, social studies, health, we’re being taught by AI practically.

And it’s very puzzling because it’s like we have teachers here who… I know for example, at my school, teachers aren’t the most fond of AI, but they do have to be in compliance and in regulation with the new laws being implemented. So, it’s puzzling that we’re seeing our teachers have to use AI and have to rely on it, when they have the certifications and they went to school to be able to teach us these skills and this knowledge.

So, yeah, I feel like it’s just being pushed out so fast without any type of pause. Where do we pause? Where do we stop?

Ariana: And for my school, we do not really… It’s a consortium school and we do not really need to use AI, because we actually work on projects and stuff like that. I think this is also a point.

I remember we had a similar kind of conversation with Chancellor Samuels, and me and some other TAP members were there, and out of all the members, majority of them were from consortium schools. And we were basically explaining the fact that how we can be able to be so much more creative in this educational system where things are more practical, rather than we just randomly memorizing stuff and just shoving them into the paper. It’s just not effective, and how we can be able to talk to you, come during the school time, because we are part of the consortium school. It’s not so much like, all the time telling us to study all the time, and stuff like that.

So, that’s how… We don’t really need all those AI in the case… like that. So, everything is slides or presentation-based, but other schools I’ve definitely heard about essays being judged by AI or checked by AI. So, you’re finding out AI by using AI, that’s kind of weird of a concept for me. So, I definitely think every school shouldn’t have this policy, because it’s just so depressing for the students itself.

Simon: Yeah. And forgive my ignorance, but what is a consortium school?

Ariana: So, consortium schools are basically those kind of schools which doesn’t really have the typical educational standards, like having APs or stuff like that. We basically do every single thing practically, like doing own research or presentations, or going in person to learn things, rather than learning it just from books or just by reading stuff.

Simon: Interesting. Well, thank you for enlightening me on that. I want to note a couple of things that… a couple of tools, AI tools, that the NYCLU… they’ve caught the NYCLU’s attention.

One is a program called GoGuardian, which has been put onto a bunch of the school computers, including personal computers for students. It’s basically monitors pretty much everything students do, every click that students make. There’s also capabilities to even supposedly monitor students, their mood, trying to monitor for problematic behavior. And also a lot of that stuff is just getting… every click, every search, et cetera, is getting scooped up and potentially fed into various data brokers or who knows where a lot of that data is going. But that’s kind of GoGuardian’s model, is in part to scoop up students’ data and profit from it. So, that’s happening.

And then we also have a blog that we’ll link to in the show notes from another TAP student who had personal experience with something called SmartPass, which I don’t know if you have that in there… Okay. Yeah, it’s a program where basically students, instead of a written regular hard copy hall pass, it’s an electronic program that teachers can actually monitor how much time someone has gone for. It monitors where the student is while they’re using the bathroom. So, it basically spies on students while they’re using the bathroom. It’s pretty dystopian.

I don’t know. Ariana, you nodded your head. Do you have any personal experience or friends who have used that?

Ariana: Yeah, definitely. As I said before, my school doesn’t really have those kind of systems, but in NYCLU, whenever… TAP meetings… whenever we used to be together, I remember there was so much conversations about it, how much they feel disgusted because it used to be four or five minutes for you to go to a bathroom and come back. And if not, then you’re going to get late, and it’s going to affect your grades and stuff like that. It’s just so much pressure on you.

And I remember we also had this summit going on in NYCLU some months ago, and we had a very specific session about AI policy in that very specific summit, and every single people in that room had problems with AI. They hated it so much. There was debates going on inside the classroom, inside the CUNY law schools. There’s this big room, and everybody was there. There was more than hundreds of students, and the session had to go over time, how much people cared about this entire issue. They were talking about how much they hate it, how much there’s smart passes going on, and stuff like that.

So, at the end of the day, the kind of surveillance going on over the students due to all this AI policies, they hate it. And whenever there’s going to be a conversation about AI among me and my peers… Some days ago, me and Yelani also went to the UN HQ just to talk about issues relating to AI, and we were having conversations with people regarding these issues. And whenever we started talking about this, there was a big conversation, and it’s just never-ending how much we absolutely hate this policy.

So, like this, it is impacting us in a very negative way, and we hope that we don’t have to fight for genuine or basic things that is supposed to be the rights of students.

Simon: Totally. And speaking of hating it, Yelani, let’s talk about… we’ve touched on some of them, but some of the problems you’ve seen come up because of the use of AI?

Yelani: Definitely. I feel as though, with AI, it isn’t the inherent fact that it’s there, because I feel like there does have to be an acceptance of, “Okay, AI isn’t going anywhere.” But I do think it is the lack of transparency surrounding AI.

Like I mentioned previously, I’ve seen AI used in a lot of my curriculum this year, and so much so to the point where we didn’t even notice it towards the end of the school year, that AI was being used and it was being normalized in our classrooms. And there’s a lack of transparency, especially in the way that it’s being promoted.

For example, there’s a program called FutureReadyNYC, which promotes itself as a program to prepare kids for career readiness and all those types of good things. And while I think that’s amazing, there’s a difference in how they’re promoting it. There’s actual classes and there’s actual field trips where students are able to hear from real mentors and to actually get that experience, but that’s not the same thing as putting AI into a classroom and having students interact with it every day. Those are two different scenarios, and I feel as though we need to be transparent on how we’re using AI.

If we’re going to use AI in the classrooms, we need to teach AI literacy. We need to teach students how to properly use it instead of just using it to complete their work, because then we have students that are submitting AI assignments and AI projects, and they don’t fully understand the work. And even when we do use AI for our curriculum and our slideshows, we think that… We have stats and data reports saying that there’s an increase in student talking time and student participation and things like that, but are students actually retaining that information? When students leave the classroom, is this information something that they’re going to remember? And if you ask most of my peers, the answer would be no.

So, I feel like transparency has been a big issue when it comes to the conversation around AI implementation.

Simon: We’ll get back to the show in just a moment, but first, if you’ve been enjoying the podcast and want to support our work, check out our merch store at shop.nyclu.org. We have a great selection of items perfect for listeners like you. Our store line lets you tell the world what you think and who you support, whether you’re marching in the streets or arguing about politics with your extended family.

And right now, you can get 20% off your order with promo code PODCAST20. That’s 20% off with promo code PODCAST20 at shop.nyclu.org. Thanks, and now back to the show.

On something you said, Yelani, it sounds like some students are using… which I’m sure listeners have heard and seen this as well… but students using things for writing assignments, right? Does that happen-

Yelani: Often.

Simon: Okay. And then you’re saying, let’s say you do that, you don’t get caught, but you’re not really learning how to write. The point of… not to sound too obvious… but a big part of writing assignments is to learn how to write, how to structure arguments, how to critically think and to express your arguments or show your research, et cetera. So, it does seem like maybe that class… if you use it, maybe that saves you some time that night, but ultimately, you’re supposed to be learning this stuff, and you might not be learning it if you’re using AI to do your assignments. Does that seem fair?

Yelani: Definitely. I feel as though, for students, it definitely can be a setback when they’re being encouraged almost to use AI as just a fast track, because at the end of the day, you might get that 100, or you might get that easy A to pass that class, but outside of that, you’re not necessarily gaining the core skills that you need to survive.

Say if we didn’t have AI, a lot of these students that are using AI now wouldn’t have those baseline foundational skills to complete assignments, and even outside of assignments, to complete everyday work on a day-to-day life skill.

So, I feel as though it’s not only setting us back, but even for the students that are completing their assignments, there are students who are getting their assignments screened as AI when they are the ones writing it. So, I think it’s just kind of a double coin in a sense, because there are students who aren’t able to learn, in a sense, because they’re not using the skills and the skills aren’t even being promoted. And then you have students who are also suffering because they’re being falsely accused of using AI, because it’s so normalized now.

Ariana: Definitely. Whenever there is some sort of AI usage there… I remember the Chair of Educational Policy member, Dinowitz… Mr. Dinowitz, he was also asking us… We went to this testimony in New York City Council some months ago, I think last month, and he was also asking us the same question at the end of our testimony, that most of the students might say that they want AI policy, or it’s going to make their education easier, and then, “Why do you guys don’t want that?”

So, we were basically talking about the fact that how the whole purpose of education, it’s to make sure that we are learning how to critical think, it’s to increase our cognitive abilities, and, as students, it’s our responsibility to understand the importance of education, because that’s the whole purpose. That’s why we learn so many hard topics every single day.

And when we are using AI, just to make sure… The name itself is pretty much self-explanatory: artificial intelligence. What kind of intelligence do I lack that I need somebody’s help, some sort of AI’s help, or I need to borrow intelligence from something else?

So, kind of like this, that we already have teachers. Teachers are so important for me. I mean, I learn stuff from them. Now, if you’re replacing AI with a teacher and trying to bring AI bots to make our education easier, it just doesn’t make sense for me. Honestly, it’s just not good for our brain at all, and it’s going to harm us ultimately in the long run. Because already we have problems in attentions and having problems in different kind of stuff, or we don’t want to do work that’s going to take much time, because who is going to work hard? Because we have AI, right?

So, we don’t want that system to be part of education. We want school to tell us that AI is not good for you, not school to tell us that AI is good for you, or you can use AI, or it’s okay to use AI.

Simon: Yeah. Okay, so we’ve talked about some problems, some concerns. I’m curious what both of your future fears are if these sorts of trends and the problems you’ve identified don’t get stopped. What are some of your fears for the future? Yelani, I’ll start with you.

Yelani: Definitely. I feel like AI, like I mentioned previously, isn’t going anywhere, as we know. So, I think a big fear for the continuation of it, if it does persist the way it is, is that it’s going to become so normalized to the point where AI is all we know, and we lack so many foundational skills that it isn’t part of the norm, and AI is solely the norm.

And I think, again, there needs to be some moderation in how fast AI is being pushed out, especially for our younger generations. I feel like we’re hearing so often that the new generations don’t know how to read, and they don’t have the proper attention span like the older generations. And it’s like, why is that? We’re not asking the question of why, and unfortunately, a lot of signs do point to AI.

So, I do fear that some of those skills and core skills that older generations and our generation were kind of brought up on aren’t going to be the same skills that younger generations are being brought up on. And I think that’s a fear of a lot of people, and it’s something that has to be addressed.

Simon: Absolutely.

Ariana: I agree. And also, I think one of the biggest issue about AI is that we don’t still know what AI is and what AI can do. There’s a lack of understanding. Our parents are not really sure how AI works and what is the use of AI. It’s such a very recent invention, and just before seeing how it actually works, bringing into the education policy… or something that basically shapes the future of a generation or future of the country… bringing a policy that is not yet well researched…

We are still not being able to… We haven’t used AI for such a long time to see what actually happens after using AI. So, without even seeing something, bringing that into the educational policy is, I think, one of the riskiest thing ever. Because imagine that it’s a failed system. After two years, we see that it’s a failed system. So, we might get rid of the failed technology, or maybe we can heal from the failed technology, but what’s going to happen with the generation whose education, entire educational life, was destroyed by that policy? We can shape or reshape a policy which was having some sort of issues, but we can’t reshape lives.

So, kind of like this, we are not educated about AI well enough to bring that into our educational system. That is my biggest fear.

Simon: I have one more dystopian example for you. So, there was a school in the Salamanca City Central School District in Western New York this fall. The district introduced this… basically like a humanoid robot in a classroom, sort of a supplement to a teacher. The district always said they wouldn’t be the teacher for students, but it would be a supplemental tool for students.

And then, after New York Focus, this New York media outlet, reported on it, there was a bunch of backlash and controversy, and the district put their plan on pause.

But I’m just curious… Ariana, I’ll start with you… what is your reaction to that, having a humanoid robot being basically a teacher assistant?

Ariana: I remember the first day I saw about this news and I was so scared that, “Oh my gosh, this is something we are literally testifying about so much. It came to Western New York. What if it comes to our city?” and stuff like that. Then after some days, I heard the policy was canceled.

And when I did a little research about why this policy was canceled, is because after the backlash, after people started protesting, they thought that, yeah, it’s kind of risky for the students’ privacy and stuff like that. So now, they took that system off.

I was so confused at that time. So, if you knew that the system had data breaches… after people protested… if somebody never protested and you still put that system into the educational policy, then you’re telling me that that entire risky system would be in the educational system if nobody protested against it? So, I was really very terrified.

And also, the price of the robot is $57,000. I mean, why do we need a teacher’s assistant, AI teacher’s assistant, which is going to be $57,000? Our teachers are amazing. I mean, they might be strict, but they’re strict not because they hate us, because they want our future to be amazing. So, it’s just weird how… $57,000 doll… and also, it’s just weird that how they only led that system to get out of the system after people protested against it.

So, the problem was from the very beginning. They just didn’t let us know about this. Again, it also brings up to the same conversation I kind of told in the previous statement. It was like, we don’t know about these dolls properly, without even knowing about these dolls or without even making it clear for the people or our parents. They’re just putting this or shoving them into the educational system without even us knowing about it. So, I hope this kind of stuff never comes to New York City, and there should be a school-wide moratorium for this entire AI or AI policy for our schools.

Yelani: I was floored. I was floored when I seen it, because we’ve already seen AI breaches in the past and data breaches in the past in our public school system.

So, I feel like it’s a little confusing that we’re not seeing repairing in the community before we’re pushing out more technology, especially when our public school students, each student gets only around 45 to $47,000 in funding, and one singular doll is $57,000 in funding.

So, why aren’t we putting more money into our students? Why aren’t we putting more money into our mental health resources, our restorative justice resources… resources that can already do the work that you’re expecting an AI bot to do?

And I think it’s definitely a question of, again, when do we pause, and then when do we actually test these different forms of technology for the better?

Simon: Totally. Okay, so as we wrap up, I just want to quickly go through some NYCLU recommendations for what to do, and then I’ll turn to you for last thoughts.

But first, as we’ve talked about and has come up over and over in this conversation, there’s just a total lack of transparency and accountability. As you both have said, almost all these technologies are implemented with very little, if any, input from parents and students, and they’re in the dark about what is being deployed in their schools. So that really needs to change. There needs to be a lot more transparency around AI in schools from New York City Public Schools and across the state.

Students and families need answers for how to opt out of AI products, that that doesn’t exist now within New York City Public Schools. The DOE has zero idea what AI technologies schools are using and what these technologies are capable of. Much of it is just sold through individual principals or superintendents. So, the NYCLU is calling for a DOE comprehensive audit to find out what’s in the schools, and for that audit to be made public so that, again, parents, teachers, students can understand what is actually being implemented.

And then also, the New York City Public Schools need to remove prohibited products from their schools. What that means is, within the guidance, there’s green light, yellow light, red light technologies, and some things have been listed as red light, meaning they shouldn’t be used. But we happen to know that many of those types of technologies are being used in New York City Public Schools, so those need to be identified and removed.

And then lastly, the AI guidance says the DOE is not yet prepared to figure out what the surveillance harms of AI technologies already are in schools today. So, given that, we are calling on New York City Public Schools to stop acquiring that kind of AI technology until it is prepared to figure out what those implications are.

So, I just want your final thoughts on what you think a good AI policy would include if the DOE came to you and asked. What would you say? I’ll start with you, Ariana.

Ariana: Definitely asking the teachers, asking the parents, asking the students about everything they want, and asking administration who generally works for students.

For example, NYCLU definitely knows what kind of situations are good for the students or what kind of situations are bad. So, stuff like that. They need to be more open to the public rather than making all the decisions without talking with people, because at the end of the day, they’re making decisions for us. So, we need to like it, and also, we need to know what kind of situations or what kind of decisions are being made for our education.

So, that’s the only request: to make sure that schools are not being controlled by AI, and every single decisions are being made by the conversation made between the students and the authority.

Yelani: Definitely. I think Ariana definitely drove that point home. The laws that are being made are for us, the students, and without having our input and our voice in those laws, the laws mean nothing, I feel.

So, laws that are transparent, laws that are explaining to students and parents and families what AI will look like in schools, what’s already happening in our schools, I think, is definitely the next step, because AI isn’t going anywhere, but we can definitely shape what it looks like in the future.

Simon: Perfect. Well, Yelani, Ariana, thank you both so much for coming on Rights This Way.

Yelani: Thank you for having us.

Ariana: Thank you.

Simon: Thank you for listening. You can find more on everything we talked about in this episode by visiting nyclu.org. And you can follow us @NYCLU on YouTube, Instagram, Bluesky, and Facebook.

Until next time, I’m Simon McCormack. Thank you for fighting with us to deliver more liberty and justice to all New Yorkers.