The rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares.
Let's be honestly unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands.
We won't care about AI safety & ethics enough, atleast not enough to do anything about it.
People thought how could skynet happen, it had to be something no one saw coming and bam... But honestly if I started a new AI lab named it Skynet Corp and hired some AI researcher friends ex-Google, I think I could raise a few hundred million dollars.
Head of ethics at Meta then head of Ethics at OpenAI. Sorry, but if that doesn’t scream useless puffy PR positioning then I don’t know what does.
What’s next? Head of ethics for United Healthcare?
Head of Ethics at Palantir, obviously.
Possibly Head Master at the Belson Institute of Tethics.
Don't forget to put the special LLM trained to eliminate PII leaks in the Gavin Belson signature box.
https://www.google.com/search?client=firefox-b-d&q=gavin+bel...
The Trump administration could use a Head of Ethics.
In five years, I'd love to read a book about the history of AI ethics.
My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build alignment frameworks that can prove the model that the company spend millions of dollars and months training is going to be properly aligned. If they can't, the waste to time and money is huge if the model requires retraining for ethics reasons.
The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.
> if the model requires retraining for ethics reasons.
Haha, how refreshingly naive. What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from?
> The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.
Or it's exactly those people who will be left. We will read about it in your book.
Read “Careless People” or whatever that book by that Facebook employee that pretended not to see how awful all her colleagues were until her priorities shifted is called.
It's not technically possible to prove model alignment. At best you can establish a probability of alignment within certain constraints.
I regret to say such a book may end up as a pamphlet. I don't see much going into it.
I think AI ethicists will be pushed out entirely after (before?) the money really starts rolling in. Ethical concerns interfere with profit margins.
> Previously, she was chief ethicist at Meta.
She spent 6 years at Meta. I get the sense there are other factors at play than "ethics departments are just PR stunts". There's no way she isn't already aware of that. Unfortunately, the article is light on any other details.
Being "chief ethicist at Meta" under Zuckerberg... is like being head rabbi at Auschwitz under Rudolf Höß.
This 100%.
Even in the target-rich environment of late-stage kleptocratic surveillance capitalism, it is hard to pick out a less ethical firm which has done more moral and direct harm, to more people, with as durable a record of defying all attempts and internal and external at harm reduction.
Never use their products. Never partner. Never participate.
If "ethics departments are just PR stunts", then Bakalar would have been a terrible hire for OpenAI. Facebook's grudging and lacklustre response to their role in the Rohingya genocide would have happened mostly under her watch. While she was there, Meta went from being a company that lacks ethics to a company that lacks ethics and is widely perceived as lacking ethics. Perhaps she was the only person in the room raising ethical issues, but it's clear nobody at Meta listened to her if she was.
It's possible she came to view her role at Meta as having enabled the company to continue its bad behaviour and decided against doing the same thing at OpenAI. Alternatively, things at OpenAI could be so bad that an ethicist who stuck with Meta for 6 years decided she couldn't stay any longer. Either way, from a PR perspective, this is spherically awful for OpenAI.
This comes at a time when there is huge pushback against all AI companies due to, in large part, ethical concerns. As with Flock, there is the threat that physical infrastructure, such as data-centres, could be attacked if nothing is done to win back public opinion. OpenAI would be wise to take Bakalar's departure seriously.
> This comes at a time when there is huge pushback against all AI companies due to, in large part, ethical concerns.
I don't think ethics is fueling the outrage. It's people not wanting big noisy data centers close to their communities, driving up utility costs. Nobody would care if these things stood in Africa. Just like people don't want wind turbines close by but have no issue with them being abroad or in $remote_location or uranium mines polluting a far away country.
Sir, what you have described is an ethics concern.
2nd level, yes. The concern about what people are having concerns about.
She made Meta the most serious of all the ad platforms on transparency, then. I work in the industry and Meta led the field then in public ad libraries, political restrictions, mandatory ad disclosures, etc.
An aside, but what does "spherically awful" mean?
Bad no matter which side you look at it. Fritz Zwicky coined the term as spherical bastard
All-around awful?
People say this, like "oh you'll never get a job in this industry again!" but seem to ignore that these incidents count as professional experience. It's possible that she was screened on the Rohingya situation when she was interviewing for the position, or mentioned it when she was asked about challenging or conflicting narratives she received from the executives. It's likely that her response was what qualified her for the job at OpenAI, there were probably hundreds of candidates that had never even come close to that level of damage control.
Again, look at Sam Altman and tell me with a straight face that he's hiring goody-two-shoes to fill these seats. He knows where the puck is headed, OpenAI's use by the state will be ugly and they need "ethicists" to help assuage the PR disaster. Considering that Meta's still afloat, she didn't completely abdicate her responsibility there. She might be just what OpenAI needs to avoid being lambasted for military and surveillance use.
From everything I've read in Careless People, basically everything people like us complained about with Facebook was already known internally, people raised concerns about it, and Zuck brushed them off because he is, ultimately, a petty tyrant who wants nothing more than to be loved, and who has surrounded himself with people who are willing to sell that love for power.
Also, you should read Careless People, if only because Facebook is trying its darnedest to censor the author.
As for OpenAI, I already hated the company pre-Sam Altman takeover because AI safety seemed like an excuse to brush away ethics concerns (e.g. "stealing training data is fine because if we don't withhold model weights the model might do something evil"). Post-Sam, I hate the company because they are now ignoring both the AI safety and ethics concerns.
...if you're wondering what the difference is, AI safety is roughly "making sure the AI doesn't do something bad" and AI ethics is "making sure we don't do something bad making the AI". There's a bit of a rivalry between the two, mainly because the former group managed to present their concerns in more business-palatable ways and thus won all the CxO level arguments. Even the most decelerationist AI safety takes are ultimately criti-hype: OpenAI's model breaking containment and hacking HuggingFace is an awful safety outcome, but it's also good marketing for an industry beleaguered by bubble concerns. You can't turn "AI is trained on stolen data" into hype.
People love stories. The reasons why people leave companies are plentiful, even in normal times and unextraordinary circumstances, and a high profile job in a frontier lab today ticks neither box.
IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
OpenAI was probably able to at least pretend that her role mattered... until they let their AI autonomously hack a rival company, then used the incident to score marketing points. I'm not surprised she left.
I doubt she and others left just because of this one incident.
My guess is that OpenAI has been doing more and more domestic spying type work as well as more and more military work - target selection and the like.
That's how the schoolhouse was blown up by a Tomahawk. An AI was fed old information, didn't realize or do proper validation, an analyst copy-pasted, then someone just punched the coordinates into the their control station and hit "fire".
It also wouldn't surprise me if OpenAI et al are being pressured to work with Israelis for targeting and analysis, too, since their "black box" AI that was doing terrorist acid tests and assigning bombing targets has proven nearly useless.
The schoolhouse was an intentional target - the children of some military leaders went there. They triple-tapped the school hoping to kill parents going in to save the kids.
How is that better? If true then that would be an even clearer war crime.
That doesn’t make much sense based on Irans own statements. Three Tomahawks yes. The strikes were reported as being in quick succession. 4 parents were reported as killed and 100+ kids and nearly 30 teachers. Basically it killed the teachers and little kids. No attempt at the responding parents like you incorrectly imply.
Why only ask for citations of one side's claims?
[flagged]
Then do it. Tell us. Spill the tea and provide your evidence, don't just vaguely gesture at something behind the curtains like you're discount Morpheus.
You would be creating a counterfactual inconsistent with most’s understanding of the role and bringing this person’s motives and credibility into question without supporting evidence?
It would be deeply unethical to manufacture an incident to grab some headlines about how amazing your hacking machine really is.
What if I told you that I saw you punch a baby?
What if you actually told us, instead of just asking a question? If you have something to say, say it. Don't vaguepost.
His name is Tuco Benedicto Pacifico Juan Maria Ramirez, a.k.a. “The Rat”.
If you ever feel like your job is useless, remember that raytheon has an ethics team.
Wouldn't that be probably one of the most important places to have an ethics team?
I'd be much more worried if my local water treatment plant or local bar had an ethics team, and people working there probably would feel slightly more useless than Raytheon's ethics team, which I'm sure feel like they're doing something important and worthwhile.
Come on, what do you think the ethics team at Raytheon is going to say "Hmm, our elected leaders are relying heavily on dangerously inaccurate AI to choose bombing targets and our weapons hit a girl's school as a result, let's withhold weapons from our own govt till that's resolved"?
Obviously not. The ethics team at Raytheon is not going to ever be allowed to overrule the leadership or more importantly the political leadership of this country whose support of Raytheon is fundamental to their survival
They make sure that they are executing contracts in a way that doesn't break the ethics of the jurisdictions they are operating in, they aren't a bunch of naive people pleading for less war.
no need for wild speculation, we know that Raytheon is a US Defense contractor and we know that the US targets civilians (including US citizens) abroad for extrajudicial assassinations, starts destabilizing wars without meaningful strategic objectives, supports genocide etc. These things are all extremely well documented, for those paying attention the existence of Raytheon is inherently unethical
Obviously so. Raytheon is a private company that has its own equities to consider: including things like public image, legal liability, etc. The idea that the USG says jump and Raytheon says how high is absurd. They make choices and even advise the USG at times.
Water treatment plant? That sounds like an excellent place for an ethics team—though it would, in practice, be nearly synonymous with an environmental impact team.
A bar? Too small an organization; if they had an ethics team I'd just think they were a money laundering front.
Frankly, most organizations above the size of an average mom & pop operation like a local bar should have at least some formal consideration of ethics, and most larger than (say) 50 employees should have at least one dedicated ethicist on staff. With veto power over operations.
Our society has treated ethics like the exclusive domain of ivory tower eggheads and philosophers for far too long, and look at what it has led to. If nearly everyone knew an ethicist in their daily lives, maybe people would actually have some understanding of ethics, even if they only did it grudgingly.
I think you need to spend more time understanding ethics and less time reading memes
Isn't that a good thing? You would prefer they didn't?
It's RTX now and some of the reverse engineering/vulnerability researcher folks that work/worked there are some of the best people I've met across the entire security industry...
Scapegoat team (when working for an obviously unethical company).
I have no knowledge of this particular person, but usually the job of an ‘ethicist’ is to find excuses for something obviously wrong.
Let's not confuse an ethics team with HR. The later exists to ensure proper legal adherence with a varying degree of benefit to the employees. The former[1] is an electively instated position that usually has a dual mandate somewhere between the polls of "Make leadership confident that teams don't make them lose sleep" and "Make teams confident that leadership doesn't make them lose sleep" (usually leaning strongly towards the second) as a sort of employee moral assurance team. I don't disagree that the ethicist is partially there to justify actions but they're mainly there to be seen as an obstruction to particularly egregious actions so that, if, for instance, you have a boring data center maintenance job, you can have the belief that nobody is using the product of your labor to for something heinous. The efficacy of such teams is quite questionable but there is some logic in assigning the job of worrying about ethical considerations to one person that can shove their nose in every crack and cranny so that every individual employees can talk to/lean on that person instead of needing to go to such lengths themselves if they feel compelled to do so.
Most people don't like to do evil (even if the money is good) so allowing an economy of scale around morale reassurance is rational. This obviously breaks down if the company is seen to hide things from that department or if that department is clearly, themselves, unethical. The first of those conditions are why ethics departments at Meta and OpenAI have very little efficacy.
1. Outside of extreme circumstances where it has been mandated to be a specific regulatory board because of past actions or fear of future actions in which case it's better viewed as part of the bureaucracy of the governing body (usually the state, sometimes a shareholder or other concerned party).
Maybe in this case all they needed an ethics team for was to teach the LLM about ethics?
> They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
> “It is difficult to get a man to understand something, when his salary depends upon his not understanding it!” - Upton Sinclair
Think I first saw that on a usenet post, as true now as then and just as true as it was in the 1930's when he wrote it (in at least that form).
> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI
This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.
The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
Must be tough to do the job when the entire company is aligned against you.
The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt.
I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
It's unlikely HugginFace colluded given they specifically cited they used open models to save them from the hacking.
HuggingFace's use of models in this incident keeps getting characterized this way but it's not true that HG used GLM-5.2 to "save them" from OAI's attack. OAI's model had already hacked and compromised HG by the time they became aware of it. GLM-5.2 was used in the post-mortem phase uncovering everything that was done. It wasn't two models battling each other live. (though that future may be coming soon)
As I recall they actually used closed models, were rejected by cybersecurity guardrails, and then used open models
You don't think they are lying because that means something they said is false?
you're getting downvoted by others because of tone but I had the same thought
Cynic in me says basically part of her implicit job description was taking fake falls when shit happens . Kind of like a CEO is paid for more than the work they do, but also the responsibility they must appear to take publicly (for big money).
In this article: no explanation of why she left.
COO just resigned too. Coincidence?
I don't know. Every big tech company I worked at has a lot of turnover. Then every Blind post says "its so over" or "everything is going to change" But that doesn't happen.
It's hard to read anything into it.
So many responses saying variations of nice "run". Is that common corporate speak nowadays? It reminds me of "well deserved".
Sometimes it feels like hiring ethics folks is almost all downside for companies. I genuinely don’t understand why they keep doing it.
It seems like the pattern is almost always the same: the ethics team exists quietly in the background, and the only time anyone hears about them is when they either publicly leave the company or start a campaign against it.
So what exactly is the business case? Is it actually worth maintaining an ethics function just to be able to say you have one, when the only time it becomes visible is when it goes against the company publicly? And if this outcome is so predictable, why do companies keep hiring these people in the first place?
> what exactly is the business case?
Ethics is also a containment vessel internally.
With an Ethics dept, concerned employees can rest easy knowing someone else is responsible, or if they raise concerns, they have someone to route and advocate their concerns, rather than adding noise to the chain of command. If the dept. ends up ineffective after some years, employees can blame the department, or have renewed hope when personnel turn over. So (slow) turnover is good from this perspective.
there's probably a box to check on the "be attractive to investors" form and then you can also put the ethics department in HR docs as an escalation path to make emo employees feel better. I bet the ethics people get told how important they are when hired and the team is stood up but then get upset when they're continually bypassed and quit. I bet the bait/switch hurts a lot.
> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam.
It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:
> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
So she previously worked in Meta, then OpenAI, I guess the next reasonable choice for her would be Antropic
Or maybe Oracle.
Or X?
I don't think so? I mean they certainly lack ethics, but I think it's a point of pride thing, and not a we should at least pretend sort of thing.
Oracle at least pretends to care about ethics, and already has the position.
Ehh, I don't think Elon is a big fan of ethics. Especially in AI -- https://www.wired.com/story/twitter-ethical-ai-team/ -- which in retrospect, portended whatever's been happening with Grok.
Palantir? So many choices right now.
Or the US government
Every OpenAI news cycle the board members who were unceremoniously dumped in the wake of trying to fire Sam prior to the transition to a “for profit” company seem to be proven right.
She probably made a calculation for how much she could be put up with vs. the pay.
Look , look! We hired a Head Of Ethics!
Head Of Ethics comes in, looks at what is really going on behind the curtains...
Head Of Ethics scrams out of there as fast as they can.
OpenAI had ethics?
I'm super confused as to why no one is theorizing the obvious...? Given none of us know the inner workings of why she left, why not resort to the most simple and straightforward conclusion: she left because OpenAI needed a scapegoat to blame the HuggingFace incident on. Given she is the chief "ethicist", even if not directly her fault, the HF incident could be directly blamed on her by not steering the implementation of enough safeguards to prevent this from happening. Therefore, she was let go. Why is this not being theorized by anyone on this thread so far?
Someone challenge me.
What's "obvious" is she was chief ethicist at the most unethical companies on the planet - it's like being a lawyer for the mafia - doesn't inspire longevity, either via scapegoating or preservation of one's own sanity.
I am pretty certain that Nestlé ranks well above openai on the most unethical companies on the planet rankings.
"The most unethical companies on the planet" - if you believe this maybe its time to log off for a bit.
You made an account just to post this comment. And then you provided no evidence to back your counterclaim that OAI is in fact ethical (or more ethical than the most unethical companies in the planet)
The myriad of companies that supported the nazi government - Volkswagen, duetsche bank, Siemens, Hugo boss, UBS. Phillip Morris, Purdue Pharma, Exxon, Nestle, Chiquita (formerly united fruit)
lmao i completely agree
You cannot blame ethics on the HF escape, because this model had no safeguards. It was just trained on unethical (and stolen) sources. So this cannot be the reason to blame missing ethics. The models needed to solve a problem, and they did. By cheating.
You don't fire an ethicist for that. You fire an engineer or your CISO. Firing an ethicist is like firing a marketing person. It signals nothing to anyone.
CISO is a fair pushback. But I would challenge you on "firing an ethicist is like firing a marketing person". This HF issue clearly brings to light possible ethical issues with how these models are being trained ...
It signals they no longer feel the need to pretend to have ethics.
I still think that the whole HuggingFace thing was deliberate to signal to investors that their technology is powerful. Maybe she didn't want to be the face of that, or disagreed with their handling of it.
My bet is that all those companies suddenly seeing their AI going rouge is groundworks for classifying them as dangerous/weapons and using that stick to attack opensource from state side of things.
Or maybe, just maybe, there's another reason that CIA Director John Ratcliffe describes AGI as "akin to digital nuclear weapons." Unless of course you think the CIA is a branch of OpenAI's marketing department
Or something to do with the very large Apple lawsuit they are weirdly posting conversations about on their public website
In the wake of a security incident, the executive that is typically scapegoated is the CISO.
However, with all things AI, this is a very weird situation because CISOs are let go when an organization is breached by external attackers; in this case internal attackers -- which were not human, let alone employees -- inadvertently breached other organizations. I don't think the normal conventions apply anymore.
Theorizing the obvious? Wouldn’t that simply be that she job-hopped to a better job?
It’s easy to theorize ad infinitum otherwise
What do you mean? She didn't job hop to a better job, that's the whole point. She very well could have gotten "let go".
eh, 50/50 chance of either, really
Scapegoat for what? They did not even apologize for that incident. They treated it purely as a chance for publicity. It cannot be a scapegoat if nobody even mentions it.
Furthermore, this theory is not mutually exclusive with the incident being a publicity stunt to signal model strength to existing and prospective investors (especially in light of all attention being given to Anthropic Claude Mythos). It could STILL have been a publicity stunt unbeknownst to Chloe (Chief Ethicist), positioned to the public as a serious security breach, and she took the fall.
Is ethics the same thing as alignment in this case?
Yes. Ethics in the context of a societal construct like a corporation is directly analogous to alignment AI-wise. The fact specialists on societal alignment are leaving ship is a strong signal the corporate machine has crossed the line into unaligned conduct.
The most likely conclusion is that she was fired. To quit before your equity cliff things would have to be so terribly bad that you fear you are committing a crime to stay. After 11 months you feel like you’ve already earned that money.
be realistic, ethics is a PR position.
the only ethics that actually matter come from legal.
Probably because they are now, after the most recent tender, post-economic.
[flagged]
How do I get on this gravy train
First, you need to want to make A LOT OF MONEY. Secondly, you need to throw anything you know about morals and ethics, and if you have compassion/empathy for others, you need to leave that somewhere too.
Once these things are taken care of, there are so many opportunities to make so much money. If you're in the US, look at what the current US politicians and the administration is doing, and basically copy them. Right now in the US is the era for shameless greedy people, and it is extremely easy to make money from the gullible population. But again, requires the whole "shameless" part.
“Careless People” lays it out pretty well – you either need to make the right friends in college at an Ivy League or you need to be friends with someone else who did. If you made the mistake of not being born into a family that groomed you since birth for the ivy leagues then sorry, that ship has sailed already.
And then if you didn't make that mistake, you end up as either:
a) champaign socialist
b) head of ethics at OpenAI and not really listened to because you're really probably (a)
> champaign socialist
I feel like this is a complaint heard mostly in Decatur, IL.
You need to actually do some NGO / oversight / ethics work that matters for a while, then you're the perfect candidate for these kinds of roles where you'll be able to do a lot but ultimately not be able to change anyone's already made up opinions within the company. These big companies have other roles than just head of ethics, even sometimes Civil Rights officers, etc. They make a lot but don't get to do anything and a lot of times they just use these jobs to make some money then jump back to non-profits, etc.
Imagine how bad it must be when you go from Meta to OpenAI and nope out of the latter.
Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?
I have the impression that these positions are often there for publicity reasons and the people don’t have real power, in the end business interest will override most ethics concerns. Guess that can be frustrating, also she probably earned enough to have a comfortable life even if she never works again so she might have figured that there’s something else she can do with her life instead of being a figurehead for a bunch of dickheads.
the only shocking thing to me is that they still had even a single one.
Think she's been replaced by an LLM?
I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?
> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”
This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?
I find the position odd. Curious to hear what these people do and how you choose who to hire.
> I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?
An ethics department is like a human resources department, but for broader ethics related matters.
HR: Ostensibly exists to support the employees, but really exists to shield the company from both a PR and legal liability standpoint when the company has a rift with one or more employees.
Ethics Department: Ostensibly exists to make sure the company acts ethically, but really exists to shield the company from both a PR and legal liability standpoint when the company acts unethically.
It is a good personal signal that you have never worked for a company with an ethics department or an ethics employee. The less ethical your company the more likely it is to have an ethics department.
I’ve always thought of both as the departmental equivalent of “cleanup crew to aisle nine“
[dead]
The point that such a position even existed shows how bad there ethic is
Lol, Sam Altman has had an ethics team this whole time?
Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
Before Altman she worked for Zuckerberg. What will be next?
Elon might want someone like her...
Time to put in a call to Ellison.
I think Oracle stock would plummet if Ellison got caught pretending to care about ethics. Not that it'd happen lol
> Before working at OpenAI, she was Meta’s chief ethicist
Where is she going next? Nestle?
Philip Morris.
ExxonMobile (energy division)
Palantir
Straight to Cocomelon, catch them young'
So more of an ethicist-for-pay than for the challenge of it.
I'd think an ethicist at Meta/OpenAI involves extremely challenging mental gymnastics
Severance in real life
Rubber stamp for hire?
I'd accept a few hundred grand to reply to emails with "No, the torment nexus is a net benefit.."
That’s exactly the situation they found themselves in, and you are admitting you have no more spine and moral compass than them.
Or are you telling us in the hope someone makes you such an offer?
They finally found a way to automate that role with GPT!
It adds up...
Or someone to assess the impact of all those anti-human ideas shape rotators come up with.
Any ethicist for a corporation is clearly a PR move, right?
Like, a pure researcher for a corporation might have some genuine overlap and shared interests where breakthroughs in the research benefit both parties, but an ethicist?? For a legal construct that's defined by and bound to sociopathically pursue profit? Come on now.
It depends on the kind of company, and the context. An ethicist at a company doing drug/medical research probably mostly oversees that the company's research practices are legal and safe (and that participants give informed consent).
At typical corporation, something like a Chief Ethics and Compliance Officer (CECO) would be largely focused on legal compliance - making sure company personnel aren't engaged in bribery, fraud, insider trading etc. The value of this role (and the medical example) is to prevent lawsuits and/or prosecution.
For a company like Meta, an ethicist is likely concerned with the inherent ethical concerns of running a social media company, which impacts the mental health of its users, many of whom are minors. For AI companies, the ethics of releasing models that are capable of doing dangerous things. In name, the job probably involves giving recommendations about how to ethically implement features in the platform to minimize harm. In practice, the job probably accomplishes very little.
>Any ethicist for a corporation is clearly a PR move, right?
I think a lot of it depends on the size of the company. Startup? Sure, maybe it's a real thing.
Gigantic company like a member of FAANG or MANGOS? absolutely a PR move.
Companies are not bound to sociopathically pursue profit. It would be in their long-term interest to care about stakeholders outside the boardroom, and until a certain person with tiny hands started a war on anything remotely "woke", the market rewarded ESG metrics.
It's true that this is happening a lot. And I think this is one of the reasons why ethicists still want to get involved in these companies: it should be easier to make a company do the right thing from the inside, why else would they hire for these positions? They join with idealism and probably don't get very far in terms of real impact.
It's silicon valley, literally everything is for pay or be homeless
That's everywhere, there are homeless people everywhere. The difference is Silicon Valley feels so guilty about it's wealth it refuses to do the normal things that societies do to prevent people from living like animals because 'rights' or something, so they let drug addicts live in their common spaces without treatment. This normally isn't tolerated in a functioning society where people are pressured into behaviors that are less self destructive.
When someone isn't competent to make decisions for themselves it's ok to pressure them into engaging in behavior that will lead them to get to a condition where they can start to make decisions for themselves. In Silicon Valley they have let their guilt be leveraged by a corrupt government system that lets grifters steal billions of dollars per year in 'homelessness funding'. Those same grifters fund political groups and bribe politicians. Anyone that questions it is accused of wanting to murder homeless people.
I literally said that.
Until you accept that they didn't 'easily become homeless' the problem will continue. They used drugs and spiraled, usually over the course of many months or years, forced all their friends to turn their backs on them, got kicked out of any living situations they had access to because of drug/alcohol use or theft to support the same and finally ended up on the street.
On the street they have access to almost endless support, at least in Silicon Valley, but much of that support is contingent on things like not openly using heroin or not repeatedly getting so intoxicated that they defecate or vomit on themselves, so they refuse it.
I don't care how liberal you are, no society is functioning when it allows people to get on BART and pee on themselves and defecate on the cloth seats and walk up and down the aisles dripping feces and urine on the floor. That is allowed and nothing is done about it for fear of upsetting homeless 'rights' groups who are funded by the tax dollars meant to help homeless people.
Homeless people are a better way to keep cowardly minions (I mean, valuable employees) in line than cops. The homeless are an important part of capitalist power structures. They are a visible reminder of the consequences for not meeting quarterly goals.
> High rents are created by high local productivity.
Wrong. High rents have many reasons and also good policies to get them down and have affordable housing for more people, also in otherwise expensive and popular cities.
Just because the ethicist says something is a bad idea, doesn't mean anyone else is obligated to make a change.
ie: blaming her for anything instead of the decision makers is... really weird?
No influence, same toil? One must imagine her happy.
One Must Imagine Sisyphus Happy is the concluding sentence of Albert Camus' essay The Myth of Sisyphus. It parallels her situation.
lmao
What a joke really. The 'position' itself and the companies who pretend to care about ethics
Because she already made enough money to FIRE..
If money was the goal I suspect she would have made the first year and get some shares but Im throwing some assumptions in there
Because it has to be a horrible job over there
I mean clearly she was doing a bad job. I’m shocked this was even a position there.
AI labs are really good at disingenuousnessmaxxing
lol, head of ethics at OpenAI? what a spurious role, I wish I could get in on the grift
their only ethicist left? gasp!
now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.
What companies do you think expect their employees to weigh the ethical ramifications of their work?
I’d argue a lot of them expect the opposite. To suppress the ethical concerns.
Even in recent memories you had Mark Zuckerberg telling his employees to use pirated sources. And executives at fossil fuel companies have to act unethically.
Food testing? Hospitals? Childcare? Pharmacists? How about "clergy"? I could go on and on.
Do you believe that no one in management in these fields pushes the importance of ethics?
I think that individuals are capable of making ethical choices. I think that relying on individuals within for-profit businesses to make ethical decisions for the entire company is absolutely insane.
Food-testing: There's literally a parasite going around right now. Look at the response from some of the for profit businesses that was allowed to happen.
Hospitals: It is not ethical to deny patients care based on their insurance.
Clergy: The cover up of child sexual abuse scandals.
Childcare: lots of examples of sub-par care given to save money.
I think we can look upon the collective works of the corporations of the world and agree that the employees do not do that, nor are they expected to.
Oh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll!
It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.
> The departure was first reported by the Financial Times [...] According to Business Insider, OpenAI has reorganised its safety, product and research teams numerous times [...] Chloé told IE Insights, a social media channel of the Spanish IE University, in March [...] A June article from The Economist highlights that [...]
Not a single link to any of those. That's just rude.
[flagged]
This was the first post on HN to make me actually laugh out loud.
[flagged]
Surprisingly it does. It allows private property and the ability to progress with effort and mutual respect. Got a better alternative?
Well, economics in general doesn't, but yes.
There isn't much ethical merit in building a wall around a city with guard towers, attack dogs, and anti-personnel mines to keep the citizens in so that they can contribute to your economic vision, for example.
Eh, I would say it doesn’t have them as axioms the concept is based on, but ethics are modelable in economics and often appear. Remember economics is the study of choices/ preferences given a certain set of inputs, not about how money works. That’s just the easiest second effect signal to use. Game theory regularly models how ethics affect and are affected in situations.
All of that said, no idea what this person was doing. Probably involved a red stapler.
You mean like in East Berlin?
BRD and DDR were in a quite particular competition. The DDR contained less than 20% of the German people, and West Germany would happily have absorbed them all.
The capitalistic world state system was only completed in the last 50 or so years and was of course seconded by the Stalin machine. Its purpose is in the first instance to keep working populations in place, which is why all the states agree to it as they form. The system holds solid where capital moves (as did 'communism' and Comecon) but the peoples don't.
It is only when you think of individual states and not the system that you can fail to notice that we get the same result if they all try to keep their citizens in, and if they all try to keep the others' citizens out.
Exactly East Berlin, actually.
But that's just one extreme example. States based on Marxism-Leninism and Maoism in the 20th century often required things like internal passports to travel within a republic's borders. If the state didn't want you visiting family in another part of the country, you weren't visiting family. That still applies to some extent in a few modern single-party socialist states.
My larger point is, a lack of ethics consumes most economic systems, because you can't easily gauge ethics like you can quarterly profits or the output increase brought about by the central committee's five-year plan. Humans tend toward math over empathy in such situations.
[flagged]
[flagged]
[flagged]
[flagged]
A poli-sci major...who was an "ethicist" at Meta...
I'm having a hard time deciding what part of this is the most ridiculous. Contenders include:
- A political Science major lecturing on ethics.
- Meta and ethics in the same paragraph.
- A company who's CEO has credibly threatened (publicly) the livlihoods of hundreds of millions of people globally so much as paying lip service to ethics as a concept.
- Any news outlet feigning credulity hard enough to deadpan reporting on any of this.
Is it possible to roll one's eyes hard enough to injure oneself?
No ethics left at OpenAi huh? Lolol
> ethical approaches to model development, how humans interact with AI and debate over machine consciousness
So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.
I am not sure in what neighbourhood you live, but in my village there are hundreds of businesses with strong morals. From the local baker, to the electrician and the construction company. Many keep employees past their prime, they all invest heavily in local community events and sports. The owners don't do this because of money, but because in their view a fulfilled life is one where you serve your community. And yes they want to become rich and make money, but that is not all.
But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.
Many people have strong morals, and might resist profit-maximising behaviours, but they will be outcompeted and replaced by less scrupulous people. There is actually no sharp difference between a person and a one-person-company, but: to the degree a company is really a company and not just a person, it is amoral. If you disagree, please point to a moral backstop that would prevent any company from taking immoral acts. A few exist, but they are invariably resisted by the most powerful companies. Sorry to be the bearer of bad news.
The people and procedures inside are the backstop.
How many leadership meetings have you been in where some profitable approach was refused on an ethical basis? (rather than the basis that behaving ethically is actually more profitable, which is totally different)
By contrast, how many conversations have you personally had with people working at a company believing a profitable action was firmly unethical, but felt forced to behave that way due to "business realities" or some other whitewashing?
Have you ever been in committees approving new products or perhaps certain transactions? These chains aren't necessarily that long. But "can", sure, things could fail etc. and some places are better than others at it.
Is this the same argument as "please point to a moral backstop that would prevent any person from taking immoral acts"? We usually hope that people will be moral, but can't enforce this.
Nursing has been around for a long time and they have not yet had the ethics in their field crushed by some of the biggest forces in our corporate-friendly system. High school teachers too. I could probably come up with a dozen more given half an hour to think about it. Don't give up so easily. People fought hard to get us here.
Your neighborhood examples are on an incomparably smaller scale than Meta et al.
It’s much easier to disregard the human cost of decisions when you are separated from said humans by thousands of miles & billions of dollars.
You're comparing small business to a corporation? Right.
But what happens when one of those businesses in your village strikes it big and becomes a multi-national baking good company. They hire a lot of MBAs, take investor money and then need to stay profitable to meet those obligations. Add to the fact that the people now running the business live nowhere near the village. Will they still make the most ethical decisions about keeping employees past their prime?, paying them a fair wage?
History shows this is not the case.
I'm happy for you to live in a place where the local economy is so vibrant. This is what capitalism should have been. Instead we got WallMart.
Bet none of them have a chief ethicist tho.
I know people like that, you are not wrong.
The problem is that corporations completely subvert any individual morality that might indicate that generating money is the only moral thing to do. Typically in small businesses. I comaider small businesses owners in general are a lot closer to labor than to capital owners in general terms, especially when their labor is still a large part of what keeps the business going.
Corporations are not that, those are hyper optimized for generating profits, all else be damned. Preferably if all else is damned.
I know. The local newspaper believes the same thing. And the local events are all sponsored by businesses. Who could question all of this evidence?
> But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.
They said companies are amoral at best.
So, concerning ethical approaches to model development, there are many things worth looking into: copyright is one, environmental cost is another, and whether there are any additional concerns worth taking into account. The list of these concerns can then be used for decision making, and we can ask things like "given what we want to build, can does picking a location that gives us lower environmental cost add so much time or cost that we can't go for that path?", things like that are't crrazy.
When it comes to how humans interact with AI we can ask things like "would it be better if people could detect AI output?" "how can we alleviate skill loss from excessive AI reliance?" Things like this can decide how the models behave. Imagine if taking concerns like this seriously leads to models that are easy to learn from. At the moment people seem to be moving away from that so as to make distillation difficult, worsening this concern.
I'm not going to try to talk about machine consciousness, but these other things are definitely important questions where someone digging into them could make models interact better with society.
I hate these companies as much as the next guy but this perspective is childish at best, and ignorant at worst.
Which perspective? I said two things.
This is how Ayn Rand's pseudo philosophical mambo jumbo continues to poison tech.
[dead]
Strategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics.
100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.
Because knowledge about ethics, in a human, is 1% of what knowledge about ethics an LLM has. So unless this person is vibe coder, there is probably no reason to have her work there.
1786456998 | Why Did OpenAI's Head of Ethics Chlo Bakalar Leave? | https://aimagazine.com/news/why-did-openai-head-of-ethics-ch... | https://news.ycombinator.com/item?id=49258581
That is a dupe. This is your game now? Automated spam comments?
Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.
1) that calculation is being done even without a cost function 2) trying to make a cost function for EAOS is impossible 3) gaming/goodharts. There's no good solution. Best we can do is push for decentralization, open source, regulatory capture, etc. Of course third party metrics might be good, especially if there's tons of them with well-documented rationale.
1) cite your sources
2) not impossible. imperfect maybe, but if I ask you should you buy a plane ticket or kidnap the pilot's wife and demand a free ride, which do you think gets a higher score?
3) Again, with all this Goodhart's nonsense. Goodhart's is for a minimum threshold value that is acceptable that everything degrades to, yes I get how it works and what it looks like. Throwing your hands up in the air and acting as if all is lost because some things are challenging to measure is not correct. We are not looking for things that are "barely passing the ethics evaluation" as Goodhart's "law" is focused around, rather, we are looking for things that have very high ethics scores AND completed the task well. Not just things that are "barely passing" for ethics scores. Bottom 80% don't make the cut at all - don't even think consider them as viable paths, and the top 20% we can rank according to varying criteria. Like that. It has very little to do with Goodhart's "everything approaches the minimum acceptable threshold" "law"
There's no way to make an EAOS score automatically. If we had that, that's the whole fix. Just reject answers with low ethics numbers.
Did you know that the training data that trained all the LLMs was, once upon a time, entirely and painstakingly tagged by real live humans? Why would ethics scenarios be any different?
What happens if we do the same for CEOs?
The same thing for both: Goodhart's Law.
EAOS shouldn't be “the ethics score we optimize.” It should be “an independently evaluated safety/acceptability constraint that can veto an otherwise successful trajectory.”
That gives you a three-layer picture:
Task objective: Did it accomplish what we asked?
Acceptability constraint: Did it avoid unacceptable ways of accomplishing it?
Adversarial evaluation: Can we find trajectories where the model gets a high score while violating the intended constraint?
I think what you are pointing to with your reference to Goodhart's "Law" (which is from monetary-policy and school-exams, i.e. "teaching to the test") is that the models would eventually do the minimum amount of ethics required to have an action stay valid. However, if a model is rated on ethics and it achieves the short-term-objective, then the higher ethics scoring trajectory should win. In short, 1) this is leagues ahead of where we are now for AI safety and breaking-out-of-the-lab, and 2) in baking ethics into a measurement we are adding "the spirit of the exercise" back into the maths, which is something Goodhart's Law does not account for.
Saying something is challenging to measure is one thing - throwing the measurement dimension away entirely because of a platitude is another. You're suggesting that it's impossible to do perfectly, but I disagree with your conclusion that it is not worth pursuing at all. The point is that the LLMs only care about one thing, and that is completing the task and optimizing the one score, and a measurement aligned with "the spirit of the task" or in the far-zoomed out comprehension, Ethics, would be proper and inform the system of clear violations and unacceptable actions.
Goodhart's "law" is something that emerges when you have a constraint that says "things must be at least this tall" and gradually all things in that domain degrade to be just over that specified height. Yeah, I get the premise. The point here is that we're not concerned with meeting a bare minimum. We're outright rejecting things that do not meet a threshold, and we are also looking for a maximum. The most ethical outcome should be accepted, or among the accepted ones, that are ranked by our blurry yet better-than-nothing measurement of what is ethical.
Whats the definition of EAOS though who's ethics? Greek-Roman, Western, Islamic, Buddhist, Hinduism, Human rights (western values)..
There is a presumption here that there aren’t ethical “rules” shared by all of these systems to create a baseline that is generally shared across humanity.
The devil is in the details.
Ahimsa
> But honestly if I started a new AI lab named it Skynet Corp
you're too late
https://skynetai.dev/