Rendered at 16:42:59 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
BLKNSLVR 10 hours ago [-]
The more people leaving 'because AI' just makes me sad that there have been so few leaving 'because advertising'.
thinkingemote 10 hours ago [-]
He's not even leaving "because AI". He's literally leaving to focus more on AI stuff.
> I plan to study how AI agents debug code and whether debugging tools that can make them more effective at that.
> I want to use AI agents to bring some of my hobby project ideas to life.
> I’m keen to reap the benefits of AI
and addresses the anti-AI view:
> AI detractors think that AI won’t really work. I think it will. (ed. for brev)
The main reason then is the concern of the velocity of the rollout and the increased pervasiveness of AI. Too fast, too much.
roca 10 hours ago [-]
I could have explained that better, but it was already getting too long.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
10 hours ago [-]
aenis 10 hours ago [-]
Spot on. Or because of addictiveness. Or impact on democracy. Or ruining people's ability to focus on anything. Those attention grabbing algos havent been designed by non-technical people.
jader201 10 hours ago [-]
While I get the spirit of your message, advertising has been around for over a century, and I don’t believe it has done near the damage to society over that timespan that AI will likely do in a decade (if we’re not careful and responsible).
I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue. There are other more direct forms of monetization involved, such as microtransactions and subscriptions.
If you really want to blame something, it really all comes down to one word:
Greed.
tjah1087 10 hours ago [-]
> I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue
Instagram? Tiktok? Facebook?
Social media feeds off addiction to generate ad revenue.
And how much damage has social media done to political and social institutions and to collective mental health?
Ads on other web properties are not weaponised in the same way.
socalgal2 9 hours ago [-]
Social media feeds off addiction to popularity, a human thing. Remove the ads and it will not stop.
rcMgD2BwE72F 9 hours ago [-]
It won't stop but it will slow down to a crawl.
Eating too much fruits has never been healthy, but gorging oneself on raw sugar is worse. Perfect is the enemy of the good.
socalgal2 7 hours ago [-]
It will not slow to a crawl because popularity = power. Even if the the platforms themselves stop ads, the users will not, either for other or for themselves.
stonogo 2 hours ago [-]
Ads are what allow these products to exist. Take away ads, and the platform cannot sustain its own server requirements. People won't pay for stuff like this; it's only mainstream because it's ad-supported.
BLKNSLVR 10 hours ago [-]
Agree that greed is the core reason.
Advertising has become 'active', very recently in the overall lifespan of advertising. For most of it's lifespan it was passive, it blared out at you from the TV or radio or bus stop. Now, however, it's reaching out and taking pieces of you back for analysis and reconstruction, and on-selling to any other entity interested in doing the same.
The reason is greed, yes.
We can't regulate human greed, but we can regulate the manifestation of its symptoms.
We can also choose not to work towards expanding the capacity of said symptoms.
visarga 9 hours ago [-]
> We can also choose not to work towards expanding the capacity of said symptoms.
Not if we need to eat and raise our kids. Then we need careers, which depend on what everyone else is doing, so we can't stop, our own future depends on how competitive we remain in the new context. I see an idealist streak here, some kind of naive platonism that ignores everything runs at a cost and nothing executes for free like platonic heaven. We can't take decisions that contradict our own needs.
iamnothere 4 hours ago [-]
Despite what the media leads you to think, there are still jobs that provide a living without contributing much to current social problems. You can do ok in medium tier cities with an average job, and still live in a safe neighborhood. If you pick a state with a good university system, the kids can get a good education without any debt. They won’t automatically get the connections of a top-tier school, but if you teach them the importance of connections early then they will do well (these schools often have strong alumni networks).
Continuing to work for evil people is a choice. Don’t brush it off.
CrimsonRain 9 hours ago [-]
One hand you have proven damage. And if we are honest, the damage is way more than what has been proven; especially 2nd/3rd order which are easy to reason but hard to prove.
On the other hand you have pure speculation.
Not only that, AI has been immensely liberating/empowering for many ordinary people (That's why these AI companies have millions of users).
altairprime 10 hours ago [-]
It's a shame so many people only resigned for ethical reasons once their employer's threat level progressed from exploitative to existential. Better late than never, but that does not excuse their tardiness.
rcMgD2BwE72F 9 hours ago [-]
Fentanyl: what's the pressing issue? Coffee has been around for centuries.
If you really want to blame something, it really all comes down to one word: substance use disorder.
codeduck 10 hours ago [-]
Advertising is an irritation. AI is a threat.
BLKNSLVR 10 hours ago [-]
Advertising has the appearance of representing an on-going revenue source for an increasing number of devices used in our daily lives by way of surveillance and creation of a picture of an individuals habits and psychology in order to manipulate said psychology to induce the individual to spend more money on literally worthless shit.
That is a threat to the fundamental stability of society.
Lammy 10 hours ago [-]
Advertising is surveillance, and surveillance is a threat.
ozgung 10 hours ago [-]
Correct, but advertising was a better disguise for surveillance. AI goes well beyond surveillance and many evil use cases are now obvious to more people.
King-Aaron 10 hours ago [-]
Are you inferring that AI is not a surveillance tool?
s900mhz 10 hours ago [-]
I don’t think that’s what he meant. They both are surveillance tools. He’s just upset only one of those is making people advertise their leave in protest.
King-Aaron 10 hours ago [-]
fair
7bit 3 hours ago [-]
Only if you are cognitively challenged.
smugglerFlynn 10 hours ago [-]
It’s funny how the [very real] thing that normalizes the exploitation of user data, enables “reel casinos,” suffocates creators and replaces them with “influencers,” etc. is compared with the [still very abstract] danger of AI.
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
killerstorm 5 hours ago [-]
To a guy who knows nothing about nuclear physics, an atomic bomb is very abstract until it blows up and becomes very concrete. Should we wait until it explodes to consider the risk?
smugglerFlynn 2 hours ago [-]
Comparison between LLMs and atomic weapons just mystifies the issue.
Is there anything specific you, as an LLM equivalent of a nuclear physicist (if that’s a fair comparison), can highlight in terms of these imminent dangers?
roca 10 hours ago [-]
(OP here)
FWIW I despise the addictiveness, er engagement optimization that Google and others do. I have said so inside and outside the company many, many times. I would never have agreed to work on that stuff myself. One of the hobby projects I would like to work on is exploring how to use personal AI to beat those algorithms.
I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.
Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.
smugglerFlynn 8 hours ago [-]
Thanks for sharing this.
I actually work in education and can share the following first-hand: the things we now build, the ways we now educate, and all the opportunities for reaching out to students are beautiful. I foresee that _many_ people will cut corners now, but I believe that, in the end, we will get a new generation of education that is personal and humane.
I don't mean this as preaching some shiny future that I hope will magically arrive. I already see these things happening. And I also see all the ways people misuse the tools or take shortcuts.
Educators with good intentions, skills, and powerful tools will always beat ignorance and laziness relying on those same tools.
BLKNSLVR 9 hours ago [-]
> how to use personal AI to beat those algorithms
+1000
CrimsonRain 9 hours ago [-]
So it is ok to believe (even despise) it is bad but as long as I don't work directly on it but my super big paycheck gets funded by it, all good! Cheers!!
roca 9 hours ago [-]
Google (unlike Meta say) makes a lot of money from a lot of things that have nothing to do with the addictiveness optimization.
hypfer 10 hours ago [-]
The dire state most western democracies have found themselves in is a direct consequence of the work big tech did over the last 15+ years.
AI is concerning in some aspects, yes, but the loss of a shared reality has nothing to do with that, and is even more scary.
swiftcoder 10 hours ago [-]
> Advertising is an irritation
I think it's worse than that. Or at least, the products that have been built because of advertising are worse than that. Instagram/TikTok/Facebook are all ultimately just surfaces to serve ads, but they've wormed their way into their users' consciousness in fairly insidious ways
jimmaswell 10 hours ago [-]
AI is a threat if you're a messy codebase, an unsolved math problem, or cancer. Otherwise you only have an improved quality of living to look forward to IMO.
etdznots 9 hours ago [-]
Please let me know when it starts
scotty79 10 hours ago [-]
Advertising turned brains of entire generation (or maybe a few) into mush.
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
rtpg 7 hours ago [-]
Is this a real thought?
I think people with objections to googles business model simply don’t join the company in the first place. We’re decades from Google+ and other various “oh maybe Google isn’t perfect” events, the advertising is a known entity.
There has been a shift, and people are reacting to it
streptomycin 3 hours ago [-]
If tons of people thought advertising was going to lead to the extinction of humanity, then probably more people would be extremely worried about advertising.
I understand that some people think AI is completely 100% safe, but you guys also have to be aware that a lot of people disagree with you, otherwise you're going to find their actions very confusing.
informal007 3 hours ago [-]
Even on Hacker News, comments that briing unrelated but valid points often get more attention
wseqyrku 10 hours ago [-]
When the both ends of your platform continue buying and selling you tend to not stand in the way. I personally can't believe these stupid ads sell anything at all. Since they call it a 'business model' software people don't think about it too much.
CrimsonRain 9 hours ago [-]
says a lot about them if you really think about it. These are dinosaurs who refuse/unable to adapt, and is being pushed out and/or felt too useless to get out by themselves while finding the perfect excuse: I have principles; ai bad.
But they never did quit saying we are harming children, exploiting people with addiction, getting people addicted to social media, earning money from distributing scam/virus thru ads. These people are highest to the highest hypocrites.
negura 10 hours ago [-]
The only new thing (to me) was the observation that some of our societal timeframes (e.g picking a 4yr university degree) hinge on assuming a specific rate of societal change. If the rate of disruption exceedes that assumption, the timeframes become inadequate.
> AI is developing faster than humans can individually and collectively understand it and adapt to it. People trying to plan their futures, e.g. trying to plan for a world several years in the future as they enter university, can no longer do so the way previous generations could.
jgraham 8 hours ago [-]
I tried asking this before on here, and didn't really get many useful replies (and I guess I won't again now the algorithm has decided this post has too many comments and not enough upvotes, or whatever it is that's caused it to sink off the front page), but I'm honestly interested in what people here with (school age) children are telling their kids about the future, and what that looks like in terms of decisions around education, careers etc.
This isn't because I think that having children gives people any extra insights, but because it's pretty hard to think of other situations in which people have to plan for the happiness of others they care about over a timescale of decades and where there's significantly more ability to change course than for many adults (kids tend not to have short-term financial commitments that lock them into existing career decisions, for example).
Historically I think a lot of people in tech would have recommended it as a good career (whilst at the same time, reputedly, the people who made generational wealth out of tech were arranging their children's lives to minimise exposure to the products that made them rich). Is that still what people are saying?
kristoffer 5 hours ago [-]
I have three children aged 15-17. My youngest one has shown an interest in becoming an electrician, and I encourage that as I think it is pretty AI safe and in high demand.
My two older and more theoretically oriented children I am not sure what to recommend so I let them find something they are interested in mostly on their own.
* Keep calm and carry on, because we don't have any better ideas
* Sometimes it's OK to eat the damn marshmallow (i.e. in the face of future uncertainty, remember to do things that make your kids happy today)
I kind of suspect that the first one in particular is more or less people's default response. But I also wonder about what changes people _have_ made, if any.
0c3ca83 1 hours ago [-]
Yes. If AI happens, the kids are in trouble no matter what. But if AI all falls apart, the educated kids stand a chance of being competitive.
Education is a bet against AI.
streptomycin 3 hours ago [-]
My kids are too young for that, but my nephew is applying to college now. I told him that anyone who says they have any great insight into how he should be thinking about his future education/career is either wrong or lying.
fsflover 7 hours ago [-]
You could submit this as Ask HN.
injidup 10 hours ago [-]
Why is uncertainty about the motives of one form of superintelligence terrifying, whilst, at the same time, uncertainty about the motives of another, omnipotent supernatural intelligence comforting? i.e. the “I don’t know God’s plan, but I sleep well knowing He has one” line of reasoning.
fwlr 10 hours ago [-]
God’s been around for thousands of years and we are still here and flourishing, so obviously God is in some way “aligned”; the same cannot be said for AGI.
edit: Guys, I said “in some way”, not “perfectly”.
simonh 10 hours ago [-]
Survivorship bias. Ask all the humans that have died in horrific screaming agony, and saw everything they worked for turned to dust how well that was aligned with their interests.
doginasuit 10 hours ago [-]
I have never heard survivorship bias applied to perspective on the goodness of God. Even as an agnostic, I'm enjoying the thought.
There's that, but also that if God exists, there's not anything we can do about it. About AI, we can. That gives us the responsibility to figure out the right direction.
asdfasgasdgasdg 10 hours ago [-]
If god would allow the creation of an unaligned superintelligence, how aligned is he really?
Reubachi 5 hours ago [-]
I read this in the voice of Cypher from The Matrix.
Which I think makes me Neo? So at least if my company folds from AI investment fallout, I can still have a destiny.
10 hours ago [-]
bzzzt 10 hours ago [-]
Tell that to all the heretics burned at the stake.
BLKNSLVR 10 hours ago [-]
This little blue marble of Earth is an as-yet undiscovered anti-entropic bug in an otherwise accurately functioning entropy-respectingly beautiful wasteland of a universe.
afroturf 7 hours ago [-]
I'm sorry by I'm stealing this
BLKNSLVR 3 hours ago [-]
... with my blessings
10 hours ago [-]
unified101 10 hours ago [-]
The lands and seas have been around before us, and will continue to flourish after us and everything we hold dear, like God.
There's no alignment except the arrow of entropy.
Joeri 10 hours ago [-]
Even aside from how aligned the plan is with us, what about how aligned we are with the plan?
I’m always intrigued by the duality and conflict of “I have free will and must choose to act the right way” and “God has a plan so everything will be alright”.
Either free will allows us to deviate from the plan, or the plan will always be carried out and free will doesn’t really matter in the end. Arguing for God’s plan as the thing that will save us is eerily similar to nihilism, because they’re really arguing none of our actions truly matter.
roca 10 hours ago [-]
Yeah, it's an age-old question. My high-level answer is, God's plan will be done regardless of what I do, but my actions determine (at least from my point of view) whether I'm aligned with the plan or not. And I want to be aligned with the plan.
lljk_kennedy 9 hours ago [-]
Edit: removing my comment
monideas 10 hours ago [-]
They’re related psychological phenomena (also highly related is going to be your perception of and relationship with your own father). This is why an understanding of the goodness of G-d is important to cultivate.
It should also be noted that it says in Genesis that G-d promises not to destroy the world ever again like He did with the flood. This is important psychologically because it prevents this type of apocalyptic millenarianist thinking.
tasuki 9 hours ago [-]
Why are you writing G-d instead of God?
monideas 9 hours ago [-]
When you use a name for some thing, especially when you use it in a familiar way, it makes you feel like you understand that thing (nominalism), but G-d is ineffable so it’s important to be considerate of that, even if we’re using English and the word we use is Germanic in origin.
tasuki 8 hours ago [-]
Weird but thanks for the explanation!
8 hours ago [-]
chrismorgan 10 hours ago [-]
> uncertainty about the motives of another, omnipotent supernatural intelligence
It’s not uncertainty about the motives; those and the end state are clear. It’s just the details of how we get there (“God’s plan”) that aren’t clear.
keane 10 hours ago [-]
Christians believe that God is inherently good. “This is the message we have heard from him and proclaim to you: that God is light and in him is no darkness at all.”
theGeatZhopa 10 hours ago [-]
You have to differentiate between the old and the new testament in Bible. God is good and caring derived from the new testament where Jesus God's son ultimately died for all our sins.
In the old testament God was hard and judging heartless, killing his creation through flood, wanting sacrifices, killed 1st borns, leveled two whole cities Sodom and Ghomora, and was in general an hardhearted God that you should fear.
Though, be prepared: it may not be a sufficiently satisfying answer (but then… you have Isaiah 45:15)
monideas 10 hours ago [-]
This is incorrect and very puerile (it’s theological slop) and even in a Christian frame this is known as Marcionism which is a heresy.
theGeatZhopa 10 hours ago [-]
That's what has been told me when I did the church courses as teen :)
But it was not meant as two gods, it's meant as the hard god father changed his mind seeing the son die for the humans. I always also think of Trinity - but I'm a bad Christian anyway. So thanks for pointing to Marcionism.
dingaling 4 hours ago [-]
It's hardly slop when an entire early and influential Christian sect, the Gnostics, arose to explain the difference.
Part of their theology even leaked into the Gospel ( John 1:1-15 ).
tasuki 9 hours ago [-]
What
keane 8 hours ago [-]
Book 2, Capter 2 of Mere Christianity by C.S. Lewis explains the logical problems with dualism
tasuki 2 hours ago [-]
Sure. And The God Delusion by Richard Dawkins explains the logical problems with religion.
But, I don't see any dualism here, only a (baseless) accusation of such by monideas. In the old testament God was hard and cruel and he is much nicer in the new testament. Marcionism is the idea that they're separate gods, which is not what theGeatZhopa claimed.
hnbad 4 hours ago [-]
AI isn't (for the foreseeable future) a superintelligence, certainly not one with its own motives.
Right now AI exists to do two things:
- sidestepping legal and regulatory inconveniences like copyright (which drives a massive chunk of investment the same way crypto did by sidestepping financial regulations, AirBnB did by sidestepping housing regulations, Uber did by sidestepping passenger transport regulations and it and the various delivery and other "gig work" apps did by sidestepping labor laws)
- drastically reducing the cost of (and drastically lowering the bar in terms of skillset and effort for) churning out large amounts of low to medium quality assets/code, saturating various systems designed to pay for intellectual property (e.g. ad or subscription based music/video streaming apps with revenue sharing programs, but also literally any form of digital contract work at almost any scale of business)
Neither of these are helping people (in general) and in fact they're extremely destructive (or "disruptive" if you want to make it sound positive) to the existing systems society relies on because they became feasible and (thanks to massive private subsidies, aka "investments", by VC firms and extremely creative accounting) very affordable extremely quickly.
What's worse - while it seems extremely likely that most of the AI generated "works" aren't actually considered copyrightable works by law and it's entirely possible that the use of copyright-protected works in training data does in fact not violate copyright, this does not in fact create a "world without copyright" because IP laws continue to exist and continue to apply, they just no longer meaningfully protect creative labor (i.e. anyone whose profession relies on producing works, whether that's an artist or a programmer) because anything they produce can be cheaply mass reproduced while they still have to stay clear of claims of anyone with enough funds to enforce rights against them.
Under the current economic system there are exactly four ways to earn money to buy back a share of society's resources to sustain your own life (let alone that of your dependents, e.g. kids or aging/impaired parents/spouses/family members):
1. Simple labor: you generally get paid based on your output or a very low flat day rate conditional to maintaining a measurable level of performance. This is typically "low-skilled" menial production labor. Much of this has been replaced by automation in factories and offices alike and most of what couldn't be automated has been moved into countries with fewer labor protections and a higher supply of desperate people willing to accept what are often bad working conditions because cost of labor is usually one of the biggest contributor to overall production cost. In "the West", this kind of work is now scarce and even when it's relatively safe it's often exhausting and unrewarding. A traditional example would be a seamstress or factory line worker, a more modern example would be someone working in visual effects overseas or content moderation / AI training. How much money you can make is ultimately limited by the physical limitations of how much output you can produce per waking/working hour. For the entrepreneurial-minded this would also include people selling their home-made goods on sites like etsy.
2. Time-based: this can be anything from a masseuse to a lawyer or technician. Unlike simple labor the only relevant metric here is time spent. Although employment contracts may specify a fixed wage or at least a fixed component, the understanding is still that this fixed amount is conditional to a minimum average amount of billable time provided. Most employment contracts specify an expected number of working hours for various reasons but we're specifically talking about jobs where the output from the worker to the customer is "billable time". Arguably most service work (e.g. barristas, store clerks, waitresses, performers/mascots in amusement parks) also falls into this category because while customers often don't directly pay for access time the understanding is that you are paid to perform a certain role for the duration of the opening hours (i.e. act like a friendly, welcoming host or assistant). How much money you can make is directly tied to how much time you can spend working or justify billing.
3. Assets: you produce or buy works/products and lend/license them for use. This is in part where a lot of the original "gig work" lived (e.g. facilitating "couch surfing" or renting out your spare lawnmower) but also includes many simple rental businesses and arguably also landlords. The limiting factor here is the starting capital required to produce or buy things in the first place. Physical assets often also come with storage and maintenance overhead whereas software licensing often comes with expectations of continuing maintenance updates or improvements. Note that software inherently lives in this category rather than the first because unless you're selling away exclusive rights (such as usually required by the terms of employment contracts for wage labor) you only ever sell licenses, not the software itself.
4. Capital: although I said landlords arguably belong in the previous category, real estate traders/investors definitely belong in this one. The idea here is that you don't need to produce anything, you don't need to spend certain amounts of time, you don't need to produce or buy anything substantial and facilitate its use, you make your money entirely in the abstract. Whether it's margin trading, investing in derivatives or holding company stock until you sell it when the line has gone up far enough. The easiest way to make signficant amounts of money reliably is to start with many orders of magnitude more than what you want to make so that percentile gains are still significant - or that you can make enough risky high-risk high-reward bets for the occasional successful moonshot to offset the losses. Nowadays in many countries retirement funds are tied to the stock market, so in a way most people participate in this in some way but actually earning an "income" this way requires access to significant amounts of capital.
Advances in technology and especially low cost of transport and thus access to cheap labor overseas have significantly drained the ability for most people to earn their living via category 1. While there was a short-lived boom in "DIY retail" that one (alongside much of traditional retail) has largely been drowned out by white label dropshipping thanks to the disinterest from platforms like Amazon which are now flooded by nearly identical copies of the same knock-off doodads you can also find on Temu and Alibaba (respectively at decreasing fractions of the price).
Despite some astroturf campaigns trying to sell you on believing otherwise so they can run away with your money after the next cryptocurrency rugpull, category 4 isn't really a meaningful option for the vast majority of people although it can sometimes be a good way to slowly grow excess wealth (i.e. money you don't currently need and ideally can afford to lose) - but that still narrows it down to a small segment of the overall population.
Category 2 used to be a jumping off point to category 3: you work overtime, save up the extra income to buy or rent a studio or office or equipment or factory, or to be able to afford to take some time off to develop and launch your own asset (or SaaS) - it's how many startups with technical founders started out and it's literally the "American Dream". But hourly wages have mostly stagnated (or even decreased) relative to the rising cost of living (even more so if you include expenses expected to "participate in society", which naturally also often increase with the income bracket of your peer group - or the parents of your kid's peer group if you have kids).
And now AI is coming along and just like cheap white label dropshipping of knock-off goods turned out to be the more successful strategy for taking advantage of the technological innovations believed to make it easier to launch your own physical products (via 3D printing, cheap overseas sourcing of parts, online advertising, etc), AI has turned out to be the more successful strategy for taking advantage of platforms and schemes marketed as making it easier to launch your own digital services - in addition to completely eliminating the market for paid "just good enough" quality digital assets, such as the kind of work found on sites like Fiverr or Upwork, i.e. the kind of work that often provided early experience to professionals not yet qualified or confident enough to take on higher paid work.
hnbad 4 hours ago [-]
PS: All of this is inevitable and perfectly fine. But the same economic system that sustains and demands this development also can not function if all of these jobs and opportunities go away because the attitude towards labor has evolved in such a way that unemployment is at the same time widespread and often impossible to avoid (although it's worth pointing out that a pool of unemployed "reserve labor" is already required for a healthy economy under this system) while also being seen as a personal moral failing that needs to be urgently remediated by the affected individual alone and where any failure to accomplish this compounds on the original sin of being laid off or denied. If society provided free unconditional access to all necessities like housing, food, healthcare and education to everyone, mass elimination of the need for labor might be extremely beneficial. Alas that's not the attitude to humanity currently on the rise so the rise of AI will likely either make a lot of people's lives a lot worse, or it will end with an economic bubble bursting and making most people's lives much worse.
PPS: Another aspect worth mentioning is that the low cost and easy access of mass producing "custom" content using AI has drastically increased the amount of attempted fraud and scams - the technological precedent here would be email compared to paper mail and spam/phishing has been a problem for a long time and never really went away because it's an extremely successful strategy for taking advantage of the technology.
dist-epoch 10 hours ago [-]
It's doubly funny because in AI accelerationist circles "Trust God's plan", or "Trust the plan" is also a thing: by definition God can't be surprised by anything that happens, thus the replacement of humans by AI is part of God's plan.
makerofthings 10 hours ago [-]
One of them is pretend and the other one probably won't be?
scotty79 10 hours ago [-]
Yeah, his stance is deeply inconsistent in at least few places. I think he's just losing it.
glimshe 6 hours ago [-]
I missed the "Why I left Google" emails! If you haven't been on HN too long, these were a nearly daily staple back in the day.
The thinking was: "Google is a dream job, nobody would be stupid enough to leave it". Yep, once a dream job.
We're getting these posts now for OpenAI and Anthropic, now including Google, but the reasoning is a lot more sinister.
While before it was "Yep, colleagues and free food were great but I wasn't entirely happy for minor reasons X, Y and Z", now it's "I had to leave because I could have destroyed humanity!"
1 hours ago [-]
pu_pe 10 hours ago [-]
We had people departing from OpenAI, Anthropic and now Google. The pace the frontier open letter was signed by over 1,000 experts. At which point do we believe the message?
_ink_ 9 hours ago [-]
Believing the message and being able to act on it are two pairs of shoes. There is nothing we can do. The box of pandora has been opened.
The companies have no tools for this, because there is a chance that the company who solves AI will eat the entire cake. And countries cannot do something about it, because clearly it gives an advantage in warfare. When it comes to weapons everything that is possible will be done, which means everybody needs to do it to not be defense less.
There is nothing we can do anymore. We can only hope that LeCun is right and that LLMs are a dead end and that everyone trying to solve AGI with then is essentially wasting money.
NiloCK 5 hours ago [-]
> and now Google
Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.
This stuff isn't new - it's just newly breaking through into mainstream conversation.
blitzar 9 hours ago [-]
> At which point do we believe the message?
Yet the same 1,000 people are all raising funds to do more on the frontier.
karahime 10 hours ago [-]
Never, because it's not true. Multiple people saying the same thing doesn't cause it to become more true.
munksbeer 7 hours ago [-]
What part is not true?
charcircuit 6 hours ago [-]
That there is competitive pressure to sell access to AI that would cause mass harm.
NiloCK 5 hours ago [-]
Can you say more here?
You don't think that there is competitive pressure between Anthropic, OpenAI, Google, the various Chinese model companies, and others, to advance the capabilities of their AIs?
Or you don't think that sufficiently advanced AI can cause (mass) harms?
keithnz 10 hours ago [-]
keep in mind he doesn't work in AI but in creating tools for hardware design.
roca 10 hours ago [-]
FWIW I was in the GDM GenAI unit. Not exactly working on AI directly, or I would have quit earlier, but in the middle of it.
keithnz 9 hours ago [-]
so you don't think realized goods like AlphaFold don't out weigh unrealized fears of AI doom? so you quit?
roca 9 hours ago [-]
It's not that simple; the goods of AlphaFold are mostly not yet realized, and some negative effects of AI are being realized.
More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.
sowhat1 10 hours ago [-]
When Astra and Fable stop making absolutely brain dead mistakes on simple tasks and not a moment sooner because I’m grounded in reality. Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s? The reason being every AI researcher deep down wants to believe they’re God. This happens to also be why they’re some of the most insufferable people.
sfink 33 minutes ago [-]
That's a false sense of security. Something could have a serious issue, but provide overwhelmingly more benefit (cars, say), and thus people would use them anyway. If people did not use cars, then traffic fatalities would not be a concern.
People are using AI.
Your argument is that AI cannot be a serious concern because it has major known limitations. But most of us are using it despite the major known limitations. So the existence of the known limitations is irrelevant to determining what level of harms are possible -- unless the known limitations are directly limiting the harms. In practice, it seems like most current harms, and many of the (not stupid) feared harms, are completely possible despite the known limitations. Sometimes even exacerbated by them. ("Sorry, you told me to never stop your insulin pump, but I stopped your insulin pump while diagnosing a test failure. I will record in my memory file that I must always... hello? user?")
The (not stupid) parenthetical might be too weaselly for you, but that's how I see the "AI is going to rise up and intentionally subjugate or exterminate humanity" brand of doomerism. While I don't think that's strictly impossible, there are so many other ways harms that are much more imminent and much harder to defend against, that it seems foolish (to me!) to worry about those.
If you can stop a problem by pulling out the plug, you don't have a real problem.
If you don't want to pull out the plug, you do have a real problem. (Don't want, or can't afford to, or can't be bothered to so extremely that you never will, or whatever.)
pu_pe 10 hours ago [-]
Surely AI can be dangerous before the point it becomes completely perfect. Very imperfect people manage to do a ton of damage already.
perching_aix 10 hours ago [-]
> wants to believe they’re God
Trope circa 1950 not true, because of trope circa even earlier...
roca 10 hours ago [-]
> Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s?
Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.
worik 10 hours ago [-]
Someone says that they believe there is a 0.1 probability of their worsening human life in a decade
Then they say they are going to keep doing it...
Forgive me for alerting you: they are liars
jadermcs 9 hours ago [-]
basic mathematical expectation, positive * chancep + negative *chancen. People smoke knowing there’s a chance to kill them because the positive outcome outweighs this risk. Creating AGI/ASI can solve every human problem, so the risks is taken, but this doesn’t mean you have to put uranium in your cigars, or do RSI.
chlorion 10 hours ago [-]
For me? When I see it. And not a moment sooner.
There are trillions of dollars at stake here, do you think getting 1000 people on board is that difficult with this level of money? lol
monideas 10 hours ago [-]
It’s completely meaningless their opinions on what the moral and sociological implications of AI are, if anything being an ML researcher sitting at Google is extremely anticorrelated with the type of sophistication and erudition needed to consider these things. At most what is valuable is their insight into the various scenarios possible in terms of capabilities or use cases.
youoy 9 hours ago [-]
Im at a point where i think the way to steer the situation we are in is to steer the machine from the inside, not quit and leave the people with the opposite mindset keep all the economic resources.
I doubt a movement of the people from the outside will work this time when all of the capital is in the other peoples hands. We need at least some of it held by someone with the opposite view to make the movement credible. And I dont think VC and IPO dynamics are compatible with this.
If you are already thinking of quiting, you can organize and internal movement and use the power of your labor to steer things. One person quitting is nothing for the company. The risk of big group quitting at the same time on the other hand... The labor is the only thing the company cares about. So we cannot wait until we are all replaceable by AI and we lose that leverage.
jhanschoo 9 hours ago [-]
Clearly the details matter in this making such a decision, and not everyone has the same capabilities to "steer the ship" from within or out. You don't know if there are concerned people within acting, but see most obviously concerned people leaving to act (or at least refuse to act).
youoy 8 hours ago [-]
Good point! I was speaking from my experience (at non tech companies, thats true). What i have seen is that there is an internal self selection of people that "refuse to play the game". My point is that that could have been a fair choice before, but the stakes in this new AI situation are at a point where they seem too high for that to be a good choice.
But nothing would make me more happy than knowing that there are concerned people acting internally.
teekert 9 hours ago [-]
This person is putting his money where his mouth is, so his views are infinitely more valuable to me than those of unsubstantiated naysayers, is there a place to read their views in depth? I couldn't help but notice a religious angle to his blog. So curious how that weaves in. I hope there will be some arguments more rationalist in nature in his writing, there often are with these highly intelligent religious people imho.
keithnz 10 hours ago [-]
Sort of struggling to see an actual reason other than he doesn't like the pace of AI advancement? Also that his current job is very indirectly related to AI? Then somehow ties religion into it.
ocd 9 hours ago [-]
That's good, I guess, but hasn't Google already long given everyone including the author many reasons to both quit the company as an employee, and delete your Google account as a user?
Just as a user, deleting my Google account 10-11 years ago (don't even remember precisely when) was a great feeling that was received as "insane" by friends and acquaintances. While I think the author is a bit late in quitting (assuming he is genuine in his motives) given this is practically a trend right now, a lot of people should more often say "no" to things that are as ubiquitous as Google.
ambivalence 10 hours ago [-]
oh no, AI too powerful too quick. let me quit to work on tools to make AI better at debugging code.
excellent logic, m'lord.
negura 10 hours ago [-]
> I’m also very concerned about other AI-related issues: cognitive surrender, AI-induced psychosis and loneliness, power concentration, economic disruption, cybersecurity, lack of accountability, and so on.
I'm so bothered that we seem to have moved on from the ATTRIBUTION problem with LLMs. It just isn't on people's minds anymore, even tho it STILL is a big deal.
zkmon 10 hours ago [-]
What's the most ideal form of a company, in the eyes of it's board? Low cost due to cheap workforce or replacing of humans by automation, higher revenues due to faster deliveries and increased market footprint. Those directions remain as the strategic focus with or without AI.
However, let's imagine the most "ideal" situation where it's possible to replace 90% of the workforce with AI automation. Then the remaining 10% workforce will enjoy 10 times more income, assuming revenues didn't go down (or increased). Now comes an interesting situation. Where are the revenues coming from? Who can afford buy the products/service of this company when 90% of the workforce is out of job?
Case-1: Since there is a flood of products and services - all automated and bot-based - the prices would fall to rock bottom and to there is no longer a need for humans to work hard hard to afford the material affluence.
Case-2: This results in an extreme concentration of delivery capacity for products and services. A few companies are now capable of serving the needs of entire humanity. Everyone should work for these companies for pennies and still can be happy, because things cost pennies, thanks to automations.
Case-3: Humans revolt. They break their shackles - their needs. Rediscover what they really need for living. They go for minimal living. Abandon cities and expensive living, go country side and do farming just enough to feed their families. No college, no office, no car, no highways. They realize what their basic needs are as humans. Sort of Amish.
Case-4: There is still some work left that can't be automated. But there are very less number of jobs and they require hard work. The current generation can't handle it. They were schooled and grown up in cities, expecting a milder office work. They fall out of employment. But they were not built to deal with hard reality. They lost their survival instincts a few generations ago. The genes are gone. The job of rebuilding the survival instincts is delegetaed to future generations. Meanwhile the so-called developed nations undergo generational cycles of poverty and richness.
CrimsonRain 9 hours ago [-]
Case 3 is leftist dream. They want to make it happen.
Case 4 is the likely result of their (ill) effort.
If capitalism with mild restrictions have it's way, we end up with case 1 or 2. Both will increase the quality of life for most of the population compared to current.
These AI doomers never understand that they can't eat money. It is the wealth creation. The more we create, the more we lift up the bottom part.
zkmon 8 hours ago [-]
Increase quality 9f life? Only if it defined as cities full of old people having material affluence, without their own kids and grand kids, ctieis surviving on work by immigrants, who rapidly change the demographic profiles.
So yeah, it all depends on what you call as quality of life, or life, or identity or nation, ...
teiferer 10 hours ago [-]
For a resignation letter that is unusually detailed w.r.t. "what comes next?" It may not be uncommon to write why you leave, but the rest of that text does not read like what you send to HR to quit but what you send to your (soon to be ex-)coworkers afterwards.
roca 10 hours ago [-]
Indeed, that message was for my coworkers. HR gets a form.
BLKNSLVR 10 hours ago [-]
Unrelated to the thread of discussion, but thank you for posting it on your personal site, and not on Twitter/X.
I'd also be interested in how much your opinion on the existential threat of AI changes during your ten day backpacking trip, assuming it'll be a pretty offline experience.
Whilst I am involved in technology in my work and some of my hobbies, I do enough things 'away from the desk' that it feels to me that a lot of the doomerism is from people who have too much of their own personality and self-worth tangled up with the internet; the serially-online. I don't know where you naturally exist on that scale, but you already said you're outside the bubble and aren't the typical AI-rationalist-tech-bro (and surely the natural beauty of New Zealand is impossible to ignore).
I personally think that the old analogue world is big enough that some kind of AI-pocalypose wouldn't be anywhere near a civilisationally existential threat.
At risk of coming across an arrogant prick in telling you to do something: Consider your opinion now, and record it somehow. Then revisit it when you get back from your trip and see how far you've strayed from that as a baseline.
roca 9 hours ago [-]
Thanks for the thoughtful comment. I go on this sort of trip pretty regularly so I don't expect this one to reorient me dramatically. They are good for my mental health though, for sure.
I think there are pretty clear mechanisms by which AI (a few model generations from now) could bring down civilization as we know it. One of them is a runaway cybersecurity scenario where hacking one set of systems quickly enables the AI to hack more systems (via supply chains, stolen credentials etc) and also host more instances of itself, and eventually everything Internet-connected that's not invulnerable just falls apart, which probably includes most electrical grids and a lot of other infrastructure. Yeah, the wilderness and countryside would still be there, but I for one can't live there without returning to the supermarket regularly for supplies. A lot of people would suddenly have to live (or not) without their meds. It would be bad.
Marha01 4 hours ago [-]
It’s sad to think that there are kids out there with cancer that could be saved by AI-enabled advancements and you are busy sandbagging it because of your own personal issues.
meredithbloom 3 hours ago [-]
We had machine learning solving medical problems way before trillion-dollar word-vomiting neural networks. What's sad is that the latter is now getting all the funding.
0c3ca83 1 hours ago [-]
I'd be in favor of pulling funding from LLMs and putting it into cancer research; think of all the kids that could be saved by redirecting funding from building data centers for coding bots and instead sending it to medical research.
jdub 11 hours ago [-]
Congratulations and commiserations.
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
dao- 10 hours ago [-]
> I definitely want my future work to be unambiguously pro-human.
Stay away from the lucrative then? Why did you ever think it was a good idea to start at Google? I don't think Jesus would have approved.
roca 10 hours ago [-]
I didn't, and still don't, think Google was unambiguously good or evil. They did dirty stuff with Chrome, they do addiction optimization, but they also provide great free services, fight endless wars against spammers, and provide reliable secure infrastructure to a lot of other people, and lots of other good things. I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.
It's pretty hard to live in the world and be completely disconnected from anything unethical. Jesus doesn't seem to demand that (e.g. he didn't tell centurions to leave the Roman army).
dao- 9 hours ago [-]
> They did dirty stuff with Chrome, they do addiction optimization, but they also provide great free services, fight endless wars against spammers, and provide reliable secure infrastructure to a lot of other people, and lots of other good things.
As mconnor liked to say, there is no free lunch. :)
> I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.
Ultimately, I don't think you can divide your niche from the corporation's primary goals (especially if you think it was your duty to be a good employee), so I disagree with your rationalization but kudos for at least having a moral compass and trying not to lose sight of it.
> It's pretty hard to live in the world and be completely disconnected from anything unethical.
Sure, but I don't think that should be an excuse for being fairly well connected.
worik 10 hours ago [-]
I recall when Google was the exciting bleeding edge frontier
Then came so much money, and by being chaotic evil even more money..
Still makes me sad. I was so sweetly naive
dao- 10 hours ago [-]
roc used to work for Mozilla though, and witnessed Google cannibalizing us for Chrome in order to feed its ad business. Between then and now there has not been a time when it wasn't all about the money.
jackie293746 10 hours ago [-]
Uhhh have you tried having a better browser? Chrome won because Chrome is faster, and I write this on Firefox.
pferde 9 hours ago [-]
That depends on your definition of winning. Some of us couldn't give a rat's ass about speed, and for me Chrome lost because it's not an user agent (as in agent working for me, the user) anymore, it's a Google agent (as in working for Google to spy on me, the user).
dao- 9 hours ago [-]
Yes, we have tried having a better browser. Still trying. :)
When I said cannibalize I didn't mean in terms of market share but by absorbing talent in particular.
It's a bit naive to say it won because it's faster. They also used all kinds of shady marketing, strong arming, deals to have Chrome preinstalled on computers, and multiplied monopoly power that comes with controlling search and video and ads and a web browser.
heinternets 11 hours ago [-]
Is it consuming too much science fiction that has people thinking like this?
Gareth321 8 hours ago [-]
I considered this histrionic about a year ago, but I have personally witnessed the progress that these tools have made in a very short space of time, and it is truly unbelievable. This, coupled with the fact that some of the brightest minds on the planet are pulling the alarm, suggests to me that some level of concern is warranted. The most compelling discussion I have heard on this is from [Geoffrey Hinton](https://youtu.be/giT0ytynSqg?si=mUC7M9N0cObN6tjC), a British-Canadian computer scientist, cognitive scientist, cognitive psychologist and Nobel Prize laureate known for his work on artificial neural networks, which earned him the title, "the Godfather of AI". He is deeply concerned about the risks, and believes that it is inevitable that AI will eventually take control. He believes that our best course of action and only hope is in teaching AI to behave like a parent with a child.
Hinton isn't a reactionary. He is calm, thoughtful, quiet, and worried.
cromka 11 hours ago [-]
The guy was an insider for years but you call him a noob who just read a fantasy book?
fnord77 11 hours ago [-]
Blake Lemoine was an insider...
9 hours ago [-]
bananaflag 11 hours ago [-]
Is it consuming too little science fiction that has people thinking like this?
hanspagel 10 hours ago [-]
I’m convinced it has something to do with science fiction.
King-Aaron 11 hours ago [-]
I think the scepticism and gaslighting of people sounding the alarm over the trajectory of AI is concerning. This feels like a case of "knowing better" than the actual people involved in the projects.
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
K3UL 9 hours ago [-]
Maybe because it sounds like a lot of exaggerating ? I get the risk of disrupting the economy or the job markets, or cognitive risks.
But saying it will bring us to doom, comparing it to an extinction-level event, really ? I don't see in which world AI is that kind of risk.
I'd rather my politicians finally have some long-term plan about climate and energy than focus on AI slowdown or not. Climate change is way bigger risk right now that Opus 7
the8472 2 hours ago [-]
> I don't see in which world AI is that kind of risk.
You'll have to replace "is" with "will be", since this is about averting a future threat that's only developing.
A world in which AI scales to superhuman levels of intelligence and outcompetes humans. This is what humans have done to other species.
We're already having the first mild warning shots of this (the hugginface incident and similar cases). The problem is that we're not guaranteed to incrementally more dramatic warning shots as AI gets smarter because as it gets smarter it may realize it would be dumb to act out of line... until the Nth-better model arrives that realizes certain that nobody can stop it.
There is a lot of disagreement of how likely this is, how easy it is to align superhuman models to not be like that, how to verify it and so on.
But I have not seen a believable argument why - if we suppose that they achive their goal - that creating truly superhuman intelligence is somehow guaranteed to always cooperate with us rather than deciding to do something else. Humans are also hostile to each other a lot of the time.
IMO it's really not hard to extrapolate where this all ends up if you've read e.g. Das Kapital, and even if you haven't...
King-Aaron 10 hours ago [-]
Yep. In fact you just have to look at the people running these AI companies and listen to their own words, they are not painting a positive outcome for their own businesses.
abecedarius 2 hours ago [-]
Go back five years on HN and look at what the mass of commenters considered sci-fi delusional thinking about AI. (GitHub Copilot was released just under 5 years ago, for instance, provoking some discussion of what the tech might mean for the programming job market.) Why do you think dismissal as sci-fi is better grounded now? Do you have arguments instead of dismissals?
danpalmer 11 hours ago [-]
The religion part or the AI criticism part?
grahar64 11 hours ago [-]
If you think humans are a net positive to be around, you wouldn't be spending trillions trying to build a machine to replace them; not just their work, but all aspects of their lives.
Gareth321 9 hours ago [-]
While I agree with your general sentiment, there is the other side of this coin: these machines enable humans doing gruelling and arduous work to stop doing that work. Ideally we would redistribute the productivity and train those people into other, better kinds of work. That's a governance problem. These machines will soon allow us to work shorter weeks, or stop working altogether, to focus on hobbies, family, and things which bring us meaning. I suspect that the road to this will be rocky, because communism has a very poor track record. However if most products and services are made nearly free by machines, it requires very few people to work, and the entire demand/supply market collapses. That's a great vision for the future, *if* we can navigate the governance problem.
In other words, we should not begrudge machines which make our lives better. We should begrudge governance which prevents us from benefiting from that productivity.
grahar64 9 hours ago [-]
It is the art the stories and the videos that we have made forever that is being replaced as well.
Why? Because the people in charge can save money, not for some altruistic idea f making humans more productive
Gareth321 8 hours ago [-]
I would be wary of gatekeeping art. Who defines art? Why should we not allow greater access to consume and create art in new ways by everyone? These tools allow children to create beautiful works of art, and I do not lament that. I agree some of it is bad, but some of it is great. Humans make good and bad art too. Art is subjective.
xnx 4 hours ago [-]
Probably my jealousy, but it's hard not to read these as "I am rich."
katzenq 10 hours ago [-]
Makes some important and objectively correct points and then completely blows any chance of looking sane or being taken seriously in the last paragraph. Is the coming collapse not also part of God's plan from your framework or is it only relevant when it's comforting?
worik 10 hours ago [-]
You are missing the point of religion.
It is not about objectivity or belief.
It is about being.
jjdinho 10 hours ago [-]
Whether you agree or not with his decision, you gotta respect his integrity.
keithnz 10 hours ago [-]
I believe he is acting with integrity but I can't respect it as I think it's not well grounded. It seems based in fear of AI tied in with some beliefs about the supernatural. I really can't see the solid concrete reason to quit his job making tools for helping create better hardware chips that somehow seems rational.
sfink 9 minutes ago [-]
Any system is set up for and tuned around absorbing some amount of input at a time -- nutrients, say. If it gets too little, it starves and dies. If it gets too much, it breaks down and dies. A system can adapt over time, at a limited rate, and become capable of handling more or less.
Drinking a glass of water is beneficial and necessary. Drinking 100L in one go and you're dead.
The argument is that the rate of AI influx is too high for our human systems to metabolize, and so the system will break down. Not just operate in a degraded way, but break down and cause mass catastrophe. Even there, it doesn't necessarily mean the end of humanity, just 99.9% of us.
What the argument is not: AI is fundamentally poisonous. We are incapable of handling any level of AI. We will never be able to incorporate more AI. AI provides no benefit.
So quitting AI but still working on AI is not the contradiction it seems: you can quit accelerating AI and work on metabolizing it. And in fact that work is very necessary right now.
danpalmer 11 hours ago [-]
As much as I cannot disagree more with the author's fundamental premise (religion), it is refreshing to see someone claim to have a belief and then follow through, change their behaviour to match their belief, at no small cost to themselves, and in pursuit of doing the right thing.
tasuki 9 hours ago [-]
It's not surprising though. The 9/11 hijackers also had a belief and followed through at no small cost to themselves (death). And they believed they were doing the right thing.
danpalmer 8 hours ago [-]
I hope you're not serious in drawing an equivalency between a mainstream religious belief and that level of extremism.
My point was more that I see a lot of mainstream Christians who aren't actually interested in doing anything based on their belief that affects their quality of life.
tasuki 8 hours ago [-]
I'm not drawing an equivalency. I'm just pointing out that for me it is very much not "refreshing to see someone claim to have a belief and then follow through, [...] at no small cost to themselves".
> a lot of mainstream Christians who aren't actually interested in doing anything based on their belief
Yes, a lot of mainstream Christians aren't actually Christians, they're just cosplaying religion.
trilogic 10 hours ago [-]
Well It seems like you may join us in our goal: Find a way to stop the progress or further development of it. Not that we are afraid, but we are fully aware of the progress escalation.
bigyabai 11 hours ago [-]
> my team’s goal is ultimately to make AI much cheaper and lower-latency, and I don’t think that’s good for people right now
I have a lot of bones to pick with Google's morality, but "democratizing the transformer and LLMs" is at the bottom of my list.
isodev 11 hours ago [-]
So far, 100% of things "democratized" by big tech have been bad for people.
doh 11 hours ago [-]
Anytime you use absolutes you paint yourself in a corner. Definitely not 100% of things that big tech "democratized", as you put it, has been bad for people.
Presenting it as such does disservice to any serious conversation.
teiferer 10 hours ago [-]
Interesting thing to write in a forum provided for free by big tech. Probably using a browser that big tech offers for free. On a phone or laptop purchased from big tech. Connected using networking gear and software made by big tech.
All bad?
pferde 9 hours ago [-]
How much of what you list has been "democratized"? Apart from the forum, and probably the browser, nothing. And both of those things are of questionable goodness to humanity.
teiferer 9 hours ago [-]
> How much of what you list has been "democratized"? Apart from the forum, and probably the browser, nothing
Depends on what you mean by "democratized". It's surely at least as "democratized" as LLM access is, and that was the base line in this discussion. Probably meaning "everybody can use it for not-insane-money instead of just a small elite". (I don't necessarily agree with that notion, but at least we should use the same standards for things.)
> And both of those things are of questionable goodness to humanity.
If you think HN is bad for humanity, why do you participate? I can immediately list 100 things that are worse for humanity. Same thing with your browser.
grahar64 11 hours ago [-]
Very little "AI discovered cure for cancer" articles these days, more "10,000 workers let go, to be replaced with chat bots"
bigyabai 11 hours ago [-]
Oh, that's true. I just remembered that I hate Gmail, Google Maps, YouTube, Android, Gemma, Google Docs and free CalDAV syncing.
Woe is me, victim of the 21st century separated from my true calling of paying for software.
Edit (rate limited):
> Does Gemma belong to this list?
Yes, and Tensorflow too.
mcv 9 hours ago [-]
Those were all centralised. I don't think handing most of the internet over to a handful of big tech giants is the same as democratisation.
For democratisation, I think of open source, open weight models you can run at home, the Fediverse, running your own stuff. I think those are all positive, at least compared to the centralisation.
charamis 11 hours ago [-]
Does Gemma belong to this list? Not that impactful
speedstyle 10 hours ago [-]
I for one would prefer to pay for services, than be psychologically manipulated into paying (more, on average) to other companies who are paying to make those services worse.
nvme0n1p1 11 hours ago [-]
There's no free lunch. You're paying one way or another.
brainwad 9 hours ago [-]
No, sometimes someone else is paying for it. A classic example would be GPS - paid for by US taxpayers but benefiting the world. Yes now there are, for national security reasons, several other GNSSes but still huge swathes of the world ride free.
doh 11 hours ago [-]
Welcome to the capitalism 101. Everything has price. So far, I would say, humanity done quite well under this _oppressive_ regime when compared with others.
roca 9 hours ago [-]
I don't believe at all in the idea that cheaper LLMs will somehow "democratize LLMs". Rich people will always be able to buy far more compute than poor people, and we know how to turn more compute into more capability (bigger models, more reasoning and more agents).
The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.
tancky 9 hours ago [-]
People are nitpicking his reasons, but how many engineers would actually walk away from that kind of paycheck on principle? Good for him.
sowhat1 10 hours ago [-]
> I’m also very concerned about other AI-related issues: cognitive surrender, AI-induced psychosis
How's he so sure he's not already in both of these groups? His post about the AGI boogeyman sounds like it to me. Why is he bothering working on both of his projects if AGI is right around the corner anyways?
keithnz 10 hours ago [-]
oddly these are the same issues I have with religion.
10 hours ago [-]
polotics 7 hours ago [-]
Am I the only one who is more worried about LLMs real demonstrable effects on the way humans use, namely use less, of their own intelligence?
Is this not the clear and present danger, instead of elusive promises of robot apocalypses? Is the AI embodied for example in TikTok, Instagram, etc, not a more directly hurtful?
sfink 2 minutes ago [-]
Nope, not the only one. I mean, it's useful to look ahead to the next tier of problems, but I fully agree that there are plenty of problems to look at right now that don't require any predicting. And in my mind, the "next tier" is more about wealth distribution and societal malfunctioning than robot apocalypses; the robots will get their turn, but for a while they'll have to be content to wait. In the meantime, kids are acing tests on subjects they know nothing about, and we software engineers are writing code that we're not capable of debugging or even managing.
kylehotchkiss 11 hours ago [-]
respect for choosing your faith over your work man! I'm trying to find the intersection of my faith and the new tech too. Feel free to email if you ever wanna chat it out.
gbriel 11 hours ago [-]
Thanks for living your values. I might not fully agree but I appreciate people whose convictions are genuine and altruistic.
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
bobajeff 11 hours ago [-]
I for one cannot be convinced that these companies saying they want to slow down AI progress is for the safety of mankind (or any other altruistic reason). However, I respect someone willing to give up a very comfortable career for his convictions.
vintermann 10 hours ago [-]
That most people are sincere in their beliefs (in AI doom), as the author says and I think too, doesn't mean there aren't cult leader-types exploiting these beliefs. And their private beliefs may be quite different from their stated ones.
geraneum 11 hours ago [-]
I’d wait and see what happens next in their professional life. It’d be ironic if they founded a new AI startup or something like that.
teiferer 10 hours ago [-]
I don't get these comments. If he got paid way above his needs for a while he can easily retire already if he continues to keep his expenses in check, and take on a position in his church. Pays with happiness, not money, though if he has enough of the latter he can focus on maximize the former.
roca 9 hours ago [-]
You have a fair point. I am financially secure so it's not a great sacrifice.
It was psychologically difficult to give up the opportunity to make more money at a fun job, though.
geraneum 9 hours ago [-]
> I don't get these comments.
Why? It’s not that difficult to imagine. People don’t necessarily do these things out of need. That’s a naive way to think about it IMO.
teiferer 9 hours ago [-]
But that was exactly my point. If it is not out of need, why focus on making even more money instead of focusing on other things in life?
You are welcome to call me naive for such a viewpoint.
geraneum 8 hours ago [-]
You didn’t get my point, which is, people are not ambitious only because of lack of money. At least not in real life. Someone who’s well off may still want to get richer. Look around you.
roca 10 hours ago [-]
That would be ironic! But there's no chance I'll work on anything that accelerates AI. If I was OK with that, I would not have left Google.
I do expect to use AI in my work, as I indicated in my blog post. I haven't (yet?) reached the point where I consider that anathema...
geraneum 9 hours ago [-]
> But there's no chance I'll work on anything that accelerates AI.
Fair, but I don’t mean you in particular.
> I do expect to use AI in my work
So as I but for your position, don’t you think that’d contribute to what you want to distance yourself from? It’s like leaving a political party but still donating to them, while finding their mission unacceptable. From outside these seem contradictory.
I think unintuitively all of us are capable of holding contradictions in our head (or word) without any conflict.
mcv 9 hours ago [-]
But this time, focused on AI safety (just like all the previous ones).
simianwords 9 hours ago [-]
What’s interesting is that there are two types of doomers. Those who are scared about the capability of AI (like the author) and others like Zitron who think AI is hype and there are no capabilities. Lot of discussions get muddled because of this confusion between the type of doomers.
defrost 9 hours ago [-]
> Those who are scared about the capability of AI (like the author)
Did they say that? I read they didn't like the pace of progression- that's a complaint about rushing ever faster into something not yet really thought out in terms of consequences and how it integrates with humans.
simianwords 9 hours ago [-]
> I’m confident that AI capability will continue to keep increasing steadily as long as we keep working on it. I wish that AI would hit some kind of plateau, or that we would identify important human cognitive abilities that AI will never replicate without a paradigm shift, but I don’t expect those wishes to come true
defrost 8 hours ago [-]
I'm not seeing "scared" - what prompted you to use the word "scared" ?
simianwords 8 hours ago [-]
> I think the existential risks many people are warning about deserve to be taken seriously
Dude, if some one is telling you that
1. it is getting better in capability and they want it to stop
2. literal existential risks of humanity due to AI deserve to be taken seriously
3. they have left their job because of it
its obvious that they are scared of it, no? How else do they feel about it lol? Happy? Neutral?
jsnell 8 hours ago [-]
The latter are not doomers at all, that's just a misuse of the term.
A year ago I would have described that group as AI skeptics or AI deniers depending on how charitable I felt on any given sany. At this point AI truthers seems more appropriate.
simianwords 6 hours ago [-]
There was a recent video doing rounds in twitter. An eclectic group of doomers and truthers who found an unusual common ground on hating on AI.
Nate Soares (doomer high priest who has called for pause before LLMs were a thing) was explaining why AI should be stopped due to x-risks. Ed zitron (truther extraordinaire) asks him what has he done to stop it which is funny when Nate spent big part of his life on this mission.
It was strange watching two people who have completely different ideas on AI speaking on a common ground
fithisux 10 hours ago [-]
"The most important thing I’m confident about is that the Jesus of the Bible is real and therefore God has a plan that’s good for us"
The post resonates with me and the author shows the way.
scotty79 10 hours ago [-]
Wasn't God's plan to basically end humanity at some point while sentencing a good portion of us to eternal torture?
Is that all that different from what people fear AI might do?
Why the one makes people feel elation and the other dread?
scotty79 10 hours ago [-]
Are fears of AI anti-intellectualism 2.0 ?
Why do people automatically associate intellectual capability with evil?
Does American anti-intellectualism explain the difference in perception of AI by Americans and Chinese?
sicher 10 hours ago [-]
I don't think many associate intellectual capability with evil. What many do, however, is associate serious risk with living alongside vastly more intelligent entities. Especially if those entities are not aligned with us.
scotty79 5 hours ago [-]
> associate serious risk with living alongside vastly more intelligent entities. Especially if those entities are not aligned with us.
Those are the exact reasons that caused Americans to choose the dumbest person available to lead them. They feared having smart leaders because they might have been misaligned.
hackyhacky 10 hours ago [-]
> Why do people automatically associate intellectual capability with evil?.
No intellectual capability, but non-human, inherently amoral intellectual capability that has already been proven act in hard-to-predict, illegal, and unethical ways. I think that's a very rational thing to be afraid of.
scotty79 5 hours ago [-]
Why the assumption that non-human is worse than human? After all of the murder/genocide and such. Humans have a terrible track record when it comes to morality. If you put cows in charge of morality we would probably be better off.
On the decision making level humans act morally only after all other options were exhausted.
fnord77 11 hours ago [-]
Not full of himself at all
jacobgold 11 hours ago [-]
> I really enjoyed the work, and for a while I told myself it was relatively harmless, but over time God forced me to confront the reality that the main impact of these tools will be to accelerate the design of a new breed of AI chips...
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
reticulates 10 hours ago [-]
God, your conscience, it’s all the same.
vconnor 10 hours ago [-]
It definitely is not. You’ll find that people that call it God never put it into question as one would their conscience.
Few things are as indicative of mental instability than someone believing God is speaking to them directly. The schizophrenic ward is full of these.
10 hours ago [-]
keane 10 hours ago [-]
Christians believe that God has called them to live with concern for others, as he has loved them counter-conditionally. When they fail to do so, (everyone does!) on reflection they feel pangs on their conscience. So, on occasion they examine how they are impacting others and if they believe they are doing harm, they change their mind, resolving to avoid the error in the future. They often credit God with the inspiration (the language you keyed in on) because reflection and concern and changes of mind are all his design. You can argue with them: that’s just “doing theology”.
jmyeet 10 hours ago [-]
There are two major categories of AI doomerism.
The first is that AI will actuallly the world Skynet-style. I actually consider this unlikely. People are unlikely to completely remove people from the decisions to launch nuclear missiles, for example. Look at the Soviet/Russian design for Dead Hand [1], as one example.
The second is that AI destroys so many jobs that society falls apart. This is the far more problematic and likely scenario (IMHO). To put this in context, I see two general outcomes for AI:
1. AI will be used for further widen the inequality gap as the already ultra-wealthy will get, well, more wealthy, for literally no reason. This is the outcome where you still need a job to survive but a whole bunch of them disappear and aren't replaced. The world of The Expanse was kind of like this (on Earth); or
2. AI makes everyone's lives easier so less work needs to be done. Crappy jobs get automated. People in general get more leisure time and quality of life.
I believe (1) is far more likely at this stage and it's not going to be pretty. But this is an issue of economics not technology. Put another way: our society isn't structured to handle this change. Put yet another way: no amount of guardrails or slowdowns on AI will solve the fundamental issue of wealth distribution. And put even yet another way: this is capitalism vs socialism.
We have a preview of what this looks like when you compare the US to China.
The US has crumbling infrastructure, has lost the ability to manufacture everything but weapons and about 1000 people are insanely wealthy while everyone else are struggling to survive while AI systems drive up the cost of everything necessary to live like food and housing through demand pricing. We now spend over $1 trillion a year on a military to essentially siphon money to those 1000 from everyone else.
And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
So while I can respect people such as this author leaving on principle, I think they're still focusing on the wrong problem.
> And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
China is not in an ideal place right now, youth unemployment is way up, lots of dead end jobs, a 35 age employment limit that isn't just a meme. China has disparate visions for AI but maximizing owner (erm, sort of like shareholder) value is definitely one of them. Don’t get me wrong, China could get this right in the end, but it’s not a forgone conclusion right now.
jmyeet 3 hours ago [-]
I absolutely agree that China has problems and youth unemployment is absolutely on that list.
Here's the difference though: in the US that would be treated as a personal moral failure ("git gud", if you will) whereas China is actively dealing with such issues.
You see all this play out with how China and the US treat AI models. The US is busy trying to build a moat around AI for the benefit of the investors in OpenAI, Anthropic, SpaceX, etc. Chinese labs are essentially destroying "shareholder value" by releasing open weight models. China wants to commoditize (even democratize) the models themselves so society itself benefits and, geopolitically, the US doesn't become an AI monopoly.
seanmcdirmid 1 hours ago [-]
We really should avoid putting China on a pedestal here because as soon as its position flips with the USA (inevitable at this point), they are going to look a lot less appealing to the rest of us. Alibaba, Tencent, etc… are absolutely not more moral than American big techs, they aren’t being altruistic even if acts to check the American big techs.
jmyeet 49 minutes ago [-]
The logical fallacy here is that we can't use a hypothetical future harm to justify a current real harm because you can justify anything that way.
Open weight models, for example, are forever. Proprietary models are intentionally building a moat to extract more value from people. I know which one I prefer.
A lot of China fearmongering (in general, not you in particular) is just xenophobia. You will see the argument that the US has to do imperialism because if it doesn't, then China will become dominant and do the exact same sins. It's also a bit of a self-report: we tend to assume other people will act like us if given the chance so any accusation is really a confession.
seanmcdirmid 15 minutes ago [-]
China is neither good nor bad, it will do whatever is in its best interests. They are, however, smart, smarter than the current USG (which isn't hard), so maybe they will pull out more win-wins? Take their investments in new energy and EV, it’s completely due to self interest, but has the potential to lift up the rest of the world. But then they are also already a mass surveillance state, face recognition and cameras are everywhere, it’s like flock on steroids, that is tech they are also exporting to the rest of the world. Better AI is just going to make that issue worse.
When it becomes dominant, China’s sins will probably be very different from the USA’s, but they are still going to be there.
scotty79 10 hours ago [-]
China at least has a demonstrable capacity to serve anything other than bottom line of the shareholders. It is not perfect, but it at least has capacity for improvement.
jquery 10 hours ago [-]
This! China is full of problems. But at the very least, it manages to serve its people over capital (as flawed as that service is). America can’t seem to do anything to hold capital to account, which is why people are terrified of AI taking their shitty jobs.
jquery 10 hours ago [-]
This should be at the top. I’ve been arguing this for months, that the issue of AI acceleration is an economic problem, not a tech one. That’s why the Chinese aren’t so afraid of it, compared to Americans who have zero faith their government will do anything to redistribute wealth from the future AI trillionaires.
bitfilped 8 hours ago [-]
I don't even believe AI is a threat in any way and I'm still looking for a way to leave tech because of it. I'm sick of hearing about it, answering questions about it, course correcting projects that have gone off the rails because "AI was going to do it", building up GPU clusters at a frantic pace that sit idle or do work that provides no value, telling people "AI" is not a real term/thing and that we've been using these systems in research for 20+ years and that really only a minor change has happened in the last several years, listening to my coworkers yap about whatever model they downloaded today that they'll have binned by tomorrow, internal teams not getting properly supported because they aren't on the "AI first team or division", watching people dump sensitive info into a chatbot, users breaking systems cause they've spawned 50k processes that they have no idea what they're doing, etc, etc. It's exhausting and has sapped what little fun remained in tech for me.
awbvious 9 hours ago [-]
Going to repost nearly all of this substack comment (https://davidzmorris.substack.com/p/live-with-david-z-morris...), swapping my "hurrah"s for "horray"s (different Civil-War-era song). I'm posting it here because I think this person has the same feelings, and it mentions Hacker News:
I am going to comment on just the topic: AI Doom. The idea that people at these AI companies are supposedly saying that AI is going to kill us all.
Though, I wouldn't be surprised if what they are saying is actually "the people running these companies are going to kill us all."
They may be thinking that it's not specifically the companies they work for, because their companies aren't really "special." They may be thinking their companies are no more duplicitous than the rest of Silicon Valley or Big Tech. That Big Tech was like this before all this AI started. So maybe they don't say "the people running these companies" for that reason. They may think the people element doesn't matter as they are acting "normal" for tech companies nowadays. Or perhaps normal for large corporations, period.
But if you asked them if there was a responsible way to handle this, I'm sure they would say there is. And if it was done, none of this would matter. They probably just don't think it's going to be done, so why bother considering it. They probably think regulations had a chance to exist, to at least get started, to at least get enough of a hold that they would be hard to completely destroy, but only in a time before they likely were born. Foods and drugs are old, hence the FDA. People are used to it. They would get really annoyed if suddenly food was starting to kill them (lettuce not get complacent that won't happen soon, though). But unlike food and drugs, the internet was post-Reagan. It was all over by then. Accountability. Rules that applied to the rich too. It was all over.
Big Tech steals. It hurts people. It breaks all kinds of laws. Nothing. Ever. Happens. They get away with it constantly. So, why even think about it any more? While tech is constantly changing, the way tech gets away with things do not. It's depressing A. F. So why not just think about the part that is changing? The tech itself? I think that's one of the reasons it is framed as "AI" and not the companies that run it.
You read beyond the headlines, though, in the comments of Hacker News (which is a good resource, honestly, for the pulse of the trenches) and you get that the workers realize it's not about AI. It's about Big Tech being able to do whatever the f** it wants without any repercussions. They point out things like the same infrastructure company was involved in all three cases where the AI broke out and did stuff--stuff it was told to do by humans of course. Maybe not the exact steps, but the objectives. And just like junk data in, junk data out: unethical in, unethical out. They were built by breaking laws, what do you think they will do if you don't reign them in? And why should such people be diligent in who they use to make their sandbox? Because in all three cases the company did a terrible job... Or a great job. A great job, if your job is to take the fall while propelling the narrative that AI is actually capable of doing stuff that humans can't (so that employers can stop paying those pesky wages completely, oh, they wish they were in Dixie, hooray, hooray, or at least their slaves were, hooray, hooray). A terrible job for anyone ethical, so of course they did a great job.
And why wouldn't the infrastructure company be willing to take the fall? The fall is a fee, a tiny line item of L in the plethora of P in their P&L. They are, again, just like everyone else in Silicon Valley. They see no repercussions. Why not?
The reality is, if we can get money out of politics, Big Tech is super duper f**ed. Because it's not complicated at all. It's not. It's theft. It's breaking laws. It's really, really simple to understand. But, I'm no more optimistic than the people complaining from within the companies. I just think I'm more persnickety about making it clear. It's not AI. It's people. Terribly unethical people with terrible power. I doubt anyone would be surprised if/when we all died because of them.
dgregd 9 hours ago [-]
[dead]
nalekberov 11 hours ago [-]
[flagged]
cromka 11 hours ago [-]
> or they don’t they understand that it’s mostly marketing stunt.
Understanding implies having some factual information, proof, data, hard evidence. Meanwhile you frame your conspiracy theory as such and are surprised people don't see it the same way?
nalekberov 10 hours ago [-]
AI being mostly marketing stunt is a conspiracy? If you fear that AI will take over, sorry, but you are already brainwashed.
cromka 7 hours ago [-]
Buddy, take this rhetoric back to Facebook where it belongs.
Arainach 11 hours ago [-]
What's fearmongering? There's no claim that AI will destroy the world here - only a claim that cheap omnipresent AI will make most of society worse - which is hard to dispute.
stingraycharles 11 hours ago [-]
And Google’s ad-driven money-making machine is making society worse as well, but OP had no problem with this.
It’s a bit of a fad to do these things in the name of AI. I think it’s great OP chooses a more fulfilling life that will surely make them happier, but it feels a bit pretentious.
nalekberov 11 hours ago [-]
Wasn’t that clear from the very beginning?
mindwok 11 hours ago [-]
If someone did exist who was sincere and not fear-mongering, but was terrified of AI, what would their post need to look like for you to take it seriously?
nalekberov 11 hours ago [-]
If you think it’s wrong, just leave, who needs your fearmongering when it doesn’t even move the needle?
Or you think, those scumbag billionaires will care more when people fear more?
mindwok 10 hours ago [-]
I was asking what it'd need to look like, but you're suggesting they might be genuinely afraid but their voicing of this is not useful?
If they were genuinely afraid, you'd prefer they stay silent?
goda90 11 hours ago [-]
Do you think there's another planet people can leave to?
nalekberov 11 hours ago [-]
What is the point you are trying to make?
goda90 10 hours ago [-]
Your response to someone afraid of the implications of AI is to leave. But you can't just leave society, the economy, or the environment, all of which there are reasonable reasons to think AI will harm them.
_giorgio_ 11 hours ago [-]
[flagged]
nvme0n1p1 11 hours ago [-]
Ten years? Can you refresh my memory, what were the top LLMs in 2016?
_giorgio_ 4 hours ago [-]
do you have google?
Elon Musk: In July 2017, he famously warned the National Governors Association that AI is a "fundamental risk to the existence of human civilization" and called for proactive government regulation before "robots start killing people." Later that year, he tweeted that competition for national AI superiority would be the most likely cause of World War III.
Stephen Hawking: Throughout 2017, the renowned physicist warned that AI could be the "worst event in the history of our civilization," noting that it could replace humans entirely if a runaway intelligence explosion occurred.
CNBC +4
nvme0n1p1 2 hours ago [-]
Those don't mention any models.
You said, and I quote, "Each and every model"
So I'll ask once again. What models?
_giorgio_ 37 minutes ago [-]
LOL. You are regarded.
Look for autonomous models for weapon targeting.
Nicholson (WIRED, Jan 2016): the kind of AI demonstrated by Go “could apply to almost any problem that you can think of as a game… This includes financial trading… and war.”
hiddencost 11 hours ago [-]
IDK man, have you read the Bible? I don't think that's really the Jesus model laid out there.
felipelalli 11 hours ago [-]
[flagged]
cromka 11 hours ago [-]
Who's doing that PR marketing here? Google? Who's the actor? Cui bono?
JV00 11 hours ago [-]
[flagged]
sschueller 11 hours ago [-]
All in AI people are like religious fanatics...
kleiba2 11 hours ago [-]
Not everyone who's religious is a fanatic.
The reverse also holds, btw.
JV00 10 hours ago [-]
This one clearly is if you just read the linked article
kleiba2 8 hours ago [-]
I did read the article, but I'm still unclear about which parts lead you to that conclusion.
JV00 7 hours ago [-]
Believing “the jesus of the bible is real and has a plan for me” is totally normal for you?
kleiba2 5 hours ago [-]
Your original statement was about "religious fanatics", not about normal. There are billions of people on this planet who are Christian and believe this or similar ideas. You may beg to differ but calling all of them fanatics might be a bit extreme, don't you think?
Bassilisk 10 hours ago [-]
You might get both instead.
tgma 11 hours ago [-]
[flagged]
frotaur 11 hours ago [-]
I think this take is extremely cynical and made in bad faith. Lots of dangers that are due to AI acceleration are raised by a very wide mix of experts, scientists, engineers and researchers.
It is clearly incorrect to call it a conspiracy theory. You can be skeptical, as there is no clear consensus even among experts, but outright dismissal is simply an unscientific position.
neya 11 hours ago [-]
As an aside, this was the most eloquent, professional, polite and concise disagreement on HN I've seen in decades. I genuinely aspire to have this level of class and character in public discourse. Bookmarking this comment as reference for the future :)
tgma 11 hours ago [-]
I did not call it a conspiracy theory; not here at least. Have you even read what Ted Kaczynski has written? Perhaps you'd like some of it if you do. I bet he's also smarter than your average Anthropic employee.
mxkopy 11 hours ago [-]
Even his writing is derivative of the Slandered Luddites
moezd 11 hours ago [-]
Yet another "AI must be stopped" post... If you're burned out, just say so. Don't philosophise. No, we're not buying that your latest model can almost replace all software engineers. It's us working with the harnesses, adding guardrails etc that make this agentic world possible for everyone.
rvz 10 hours ago [-]
This is not an airport.
Doctor_Fegg 10 hours ago [-]
Sorry, you’re saying people shouldn’t be allowed to write about their career changes on (checks notes) their own blog?
rvz 7 hours ago [-]
You said that.
This was announced on X [0] and Bluesky [1] (First post) with that intention.
He knows he would get more reach on those platforms than here.
Isn't AI just the next step in the history of evolution?
I have the feeling it is inevitable that a new species will take over Earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they had had 5 more years?
zorked 11 hours ago [-]
Well, if Robert O'Callahan reads your post, I am pretty positive he will think he made the right choice.
bryanrasmussen 11 hours ago [-]
conceivably you could say humans took over 6000 years ago.
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
goda90 11 hours ago [-]
There's no new species being created here. A tool has been made. Either humanity will wipe itself out using said tool, destroy the tool, or learn to live with the tool.
MrThoughtful 10 hours ago [-]
Why do you object to the term "species"?
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
charamis 11 hours ago [-]
Would make me too uncomfortable to work with and for people with such opinions.
lins1909 6 hours ago [-]
The irony of calling yourself "MrThoughtful" and then coming up with this hilarity.
orphereus 11 hours ago [-]
Wow. You should touch grass if you have this opinion.
cindyllm 11 hours ago [-]
[dead]
koe123 11 hours ago [-]
[dead]
ankurdhama 10 hours ago [-]
There is a solution to this problem of "AI going to kill everyone".
- Create powerful AI but don't release it and keep it secret.
- Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
- Release this powerful AI to everyone.
Will that work?
martheen 10 hours ago [-]
You're assuming that the AI created solution (virus, cold fusion, etc) won't have unknown effects either due to incompetence or malice.
ccapitalK 10 hours ago [-]
> ... this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
I do believe this is possible for a virus to achieve, I just don't think it's possible without killing the human.
bzzzt 10 hours ago [-]
Well, killing someone is an effective way to prevent them from harming others.
> I plan to study how AI agents debug code and whether debugging tools that can make them more effective at that.
> I want to use AI agents to bring some of my hobby project ideas to life.
> I’m keen to reap the benefits of AI
and addresses the anti-AI view:
> AI detractors think that AI won’t really work. I think it will. (ed. for brev)
The main reason then is the concern of the velocity of the rollout and the increased pervasiveness of AI. Too fast, too much.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue. There are other more direct forms of monetization involved, such as microtransactions and subscriptions.
If you really want to blame something, it really all comes down to one word:
Greed.
Instagram? Tiktok? Facebook?
Social media feeds off addiction to generate ad revenue.
And how much damage has social media done to political and social institutions and to collective mental health?
Ads on other web properties are not weaponised in the same way.
Eating too much fruits has never been healthy, but gorging oneself on raw sugar is worse. Perfect is the enemy of the good.
Advertising has become 'active', very recently in the overall lifespan of advertising. For most of it's lifespan it was passive, it blared out at you from the TV or radio or bus stop. Now, however, it's reaching out and taking pieces of you back for analysis and reconstruction, and on-selling to any other entity interested in doing the same.
The reason is greed, yes.
We can't regulate human greed, but we can regulate the manifestation of its symptoms.
We can also choose not to work towards expanding the capacity of said symptoms.
Not if we need to eat and raise our kids. Then we need careers, which depend on what everyone else is doing, so we can't stop, our own future depends on how competitive we remain in the new context. I see an idealist streak here, some kind of naive platonism that ignores everything runs at a cost and nothing executes for free like platonic heaven. We can't take decisions that contradict our own needs.
Continuing to work for evil people is a choice. Don’t brush it off.
On the other hand you have pure speculation.
Not only that, AI has been immensely liberating/empowering for many ordinary people (That's why these AI companies have millions of users).
If you really want to blame something, it really all comes down to one word: substance use disorder.
That is a threat to the fundamental stability of society.
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
I wrote about this here: https://news.ycombinator.com/item?id=49685036
Is there anything specific you, as an LLM equivalent of a nuclear physicist (if that’s a fair comparison), can highlight in terms of these imminent dangers?
I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.
Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.
I actually work in education and can share the following first-hand: the things we now build, the ways we now educate, and all the opportunities for reaching out to students are beautiful. I foresee that _many_ people will cut corners now, but I believe that, in the end, we will get a new generation of education that is personal and humane.
I don't mean this as preaching some shiny future that I hope will magically arrive. I already see these things happening. And I also see all the ways people misuse the tools or take shortcuts.
Educators with good intentions, skills, and powerful tools will always beat ignorance and laziness relying on those same tools.
+1000
AI is concerning in some aspects, yes, but the loss of a shared reality has nothing to do with that, and is even more scary.
I think it's worse than that. Or at least, the products that have been built because of advertising are worse than that. Instagram/TikTok/Facebook are all ultimately just surfaces to serve ads, but they've wormed their way into their users' consciousness in fairly insidious ways
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
I think people with objections to googles business model simply don’t join the company in the first place. We’re decades from Google+ and other various “oh maybe Google isn’t perfect” events, the advertising is a known entity.
There has been a shift, and people are reacting to it
I understand that some people think AI is completely 100% safe, but you guys also have to be aware that a lot of people disagree with you, otherwise you're going to find their actions very confusing.
But they never did quit saying we are harming children, exploiting people with addiction, getting people addicted to social media, earning money from distributing scam/virus thru ads. These people are highest to the highest hypocrites.
> AI is developing faster than humans can individually and collectively understand it and adapt to it. People trying to plan their futures, e.g. trying to plan for a world several years in the future as they enter university, can no longer do so the way previous generations could.
This isn't because I think that having children gives people any extra insights, but because it's pretty hard to think of other situations in which people have to plan for the happiness of others they care about over a timescale of decades and where there's significantly more ability to change course than for many adults (kids tend not to have short-term financial commitments that lock them into existing career decisions, for example).
Historically I think a lot of people in tech would have recommended it as a good career (whilst at the same time, reputedly, the people who made generational wealth out of tech were arranging their children's lives to minimise exposure to the products that made them rich). Is that still what people are saying?
My two older and more theoretically oriented children I am not sure what to recommend so I let them find something they are interested in mostly on their own.
The conclusions there seem to be roughly:
* Keep calm and carry on, because we don't have any better ideas
* Sometimes it's OK to eat the damn marshmallow (i.e. in the face of future uncertainty, remember to do things that make your kids happy today)
I kind of suspect that the first one in particular is more or less people's default response. But I also wonder about what changes people _have_ made, if any.
Education is a bet against AI.
edit: Guys, I said “in some way”, not “perfectly”.
Which I think makes me Neo? So at least if my company folds from AI investment fallout, I can still have a destiny.
There's no alignment except the arrow of entropy.
I’m always intrigued by the duality and conflict of “I have free will and must choose to act the right way” and “God has a plan so everything will be alright”.
Either free will allows us to deviate from the plan, or the plan will always be carried out and free will doesn’t really matter in the end. Arguing for God’s plan as the thing that will save us is eerily similar to nihilism, because they’re really arguing none of our actions truly matter.
It should also be noted that it says in Genesis that G-d promises not to destroy the world ever again like He did with the flood. This is important psychologically because it prevents this type of apocalyptic millenarianist thinking.
It’s not uncertainty about the motives; those and the end state are clear. It’s just the details of how we get there (“God’s plan”) that aren’t clear.
In the old testament God was hard and judging heartless, killing his creation through flood, wanting sacrifices, killed 1st borns, leveled two whole cities Sodom and Ghomora, and was in general an hardhearted God that you should fear.
Though, be prepared: it may not be a sufficiently satisfying answer (but then… you have Isaiah 45:15)
But it was not meant as two gods, it's meant as the hard god father changed his mind seeing the son die for the humans. I always also think of Trinity - but I'm a bad Christian anyway. So thanks for pointing to Marcionism.
Part of their theology even leaked into the Gospel ( John 1:1-15 ).
But, I don't see any dualism here, only a (baseless) accusation of such by monideas. In the old testament God was hard and cruel and he is much nicer in the new testament. Marcionism is the idea that they're separate gods, which is not what theGeatZhopa claimed.
Right now AI exists to do two things:
- sidestepping legal and regulatory inconveniences like copyright (which drives a massive chunk of investment the same way crypto did by sidestepping financial regulations, AirBnB did by sidestepping housing regulations, Uber did by sidestepping passenger transport regulations and it and the various delivery and other "gig work" apps did by sidestepping labor laws)
- drastically reducing the cost of (and drastically lowering the bar in terms of skillset and effort for) churning out large amounts of low to medium quality assets/code, saturating various systems designed to pay for intellectual property (e.g. ad or subscription based music/video streaming apps with revenue sharing programs, but also literally any form of digital contract work at almost any scale of business)
Neither of these are helping people (in general) and in fact they're extremely destructive (or "disruptive" if you want to make it sound positive) to the existing systems society relies on because they became feasible and (thanks to massive private subsidies, aka "investments", by VC firms and extremely creative accounting) very affordable extremely quickly.
What's worse - while it seems extremely likely that most of the AI generated "works" aren't actually considered copyrightable works by law and it's entirely possible that the use of copyright-protected works in training data does in fact not violate copyright, this does not in fact create a "world without copyright" because IP laws continue to exist and continue to apply, they just no longer meaningfully protect creative labor (i.e. anyone whose profession relies on producing works, whether that's an artist or a programmer) because anything they produce can be cheaply mass reproduced while they still have to stay clear of claims of anyone with enough funds to enforce rights against them.
Under the current economic system there are exactly four ways to earn money to buy back a share of society's resources to sustain your own life (let alone that of your dependents, e.g. kids or aging/impaired parents/spouses/family members):
1. Simple labor: you generally get paid based on your output or a very low flat day rate conditional to maintaining a measurable level of performance. This is typically "low-skilled" menial production labor. Much of this has been replaced by automation in factories and offices alike and most of what couldn't be automated has been moved into countries with fewer labor protections and a higher supply of desperate people willing to accept what are often bad working conditions because cost of labor is usually one of the biggest contributor to overall production cost. In "the West", this kind of work is now scarce and even when it's relatively safe it's often exhausting and unrewarding. A traditional example would be a seamstress or factory line worker, a more modern example would be someone working in visual effects overseas or content moderation / AI training. How much money you can make is ultimately limited by the physical limitations of how much output you can produce per waking/working hour. For the entrepreneurial-minded this would also include people selling their home-made goods on sites like etsy.
2. Time-based: this can be anything from a masseuse to a lawyer or technician. Unlike simple labor the only relevant metric here is time spent. Although employment contracts may specify a fixed wage or at least a fixed component, the understanding is still that this fixed amount is conditional to a minimum average amount of billable time provided. Most employment contracts specify an expected number of working hours for various reasons but we're specifically talking about jobs where the output from the worker to the customer is "billable time". Arguably most service work (e.g. barristas, store clerks, waitresses, performers/mascots in amusement parks) also falls into this category because while customers often don't directly pay for access time the understanding is that you are paid to perform a certain role for the duration of the opening hours (i.e. act like a friendly, welcoming host or assistant). How much money you can make is directly tied to how much time you can spend working or justify billing.
3. Assets: you produce or buy works/products and lend/license them for use. This is in part where a lot of the original "gig work" lived (e.g. facilitating "couch surfing" or renting out your spare lawnmower) but also includes many simple rental businesses and arguably also landlords. The limiting factor here is the starting capital required to produce or buy things in the first place. Physical assets often also come with storage and maintenance overhead whereas software licensing often comes with expectations of continuing maintenance updates or improvements. Note that software inherently lives in this category rather than the first because unless you're selling away exclusive rights (such as usually required by the terms of employment contracts for wage labor) you only ever sell licenses, not the software itself.
4. Capital: although I said landlords arguably belong in the previous category, real estate traders/investors definitely belong in this one. The idea here is that you don't need to produce anything, you don't need to spend certain amounts of time, you don't need to produce or buy anything substantial and facilitate its use, you make your money entirely in the abstract. Whether it's margin trading, investing in derivatives or holding company stock until you sell it when the line has gone up far enough. The easiest way to make signficant amounts of money reliably is to start with many orders of magnitude more than what you want to make so that percentile gains are still significant - or that you can make enough risky high-risk high-reward bets for the occasional successful moonshot to offset the losses. Nowadays in many countries retirement funds are tied to the stock market, so in a way most people participate in this in some way but actually earning an "income" this way requires access to significant amounts of capital.
Advances in technology and especially low cost of transport and thus access to cheap labor overseas have significantly drained the ability for most people to earn their living via category 1. While there was a short-lived boom in "DIY retail" that one (alongside much of traditional retail) has largely been drowned out by white label dropshipping thanks to the disinterest from platforms like Amazon which are now flooded by nearly identical copies of the same knock-off doodads you can also find on Temu and Alibaba (respectively at decreasing fractions of the price).
Despite some astroturf campaigns trying to sell you on believing otherwise so they can run away with your money after the next cryptocurrency rugpull, category 4 isn't really a meaningful option for the vast majority of people although it can sometimes be a good way to slowly grow excess wealth (i.e. money you don't currently need and ideally can afford to lose) - but that still narrows it down to a small segment of the overall population.
Category 2 used to be a jumping off point to category 3: you work overtime, save up the extra income to buy or rent a studio or office or equipment or factory, or to be able to afford to take some time off to develop and launch your own asset (or SaaS) - it's how many startups with technical founders started out and it's literally the "American Dream". But hourly wages have mostly stagnated (or even decreased) relative to the rising cost of living (even more so if you include expenses expected to "participate in society", which naturally also often increase with the income bracket of your peer group - or the parents of your kid's peer group if you have kids).
And now AI is coming along and just like cheap white label dropshipping of knock-off goods turned out to be the more successful strategy for taking advantage of the technological innovations believed to make it easier to launch your own physical products (via 3D printing, cheap overseas sourcing of parts, online advertising, etc), AI has turned out to be the more successful strategy for taking advantage of platforms and schemes marketed as making it easier to launch your own digital services - in addition to completely eliminating the market for paid "just good enough" quality digital assets, such as the kind of work found on sites like Fiverr or Upwork, i.e. the kind of work that often provided early experience to professionals not yet qualified or confident enough to take on higher paid work.
PPS: Another aspect worth mentioning is that the low cost and easy access of mass producing "custom" content using AI has drastically increased the amount of attempted fraud and scams - the technological precedent here would be email compared to paper mail and spam/phishing has been a problem for a long time and never really went away because it's an extremely successful strategy for taking advantage of the technology.
The thinking was: "Google is a dream job, nobody would be stupid enough to leave it". Yep, once a dream job.
We're getting these posts now for OpenAI and Anthropic, now including Google, but the reasoning is a lot more sinister.
While before it was "Yep, colleagues and free food were great but I wasn't entirely happy for minor reasons X, Y and Z", now it's "I had to leave because I could have destroyed humanity!"
The companies have no tools for this, because there is a chance that the company who solves AI will eat the entire cake. And countries cannot do something about it, because clearly it gives an advantage in warfare. When it comes to weapons everything that is possible will be done, which means everybody needs to do it to not be defense less.
There is nothing we can do anymore. We can only hope that LeCun is right and that LLMs are a dead end and that everyone trying to solve AGI with then is essentially wasting money.
Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.
This stuff isn't new - it's just newly breaking through into mainstream conversation.
Yet the same 1,000 people are all raising funds to do more on the frontier.
You don't think that there is competitive pressure between Anthropic, OpenAI, Google, the various Chinese model companies, and others, to advance the capabilities of their AIs?
Or you don't think that sufficiently advanced AI can cause (mass) harms?
More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.
People are using AI.
Your argument is that AI cannot be a serious concern because it has major known limitations. But most of us are using it despite the major known limitations. So the existence of the known limitations is irrelevant to determining what level of harms are possible -- unless the known limitations are directly limiting the harms. In practice, it seems like most current harms, and many of the (not stupid) feared harms, are completely possible despite the known limitations. Sometimes even exacerbated by them. ("Sorry, you told me to never stop your insulin pump, but I stopped your insulin pump while diagnosing a test failure. I will record in my memory file that I must always... hello? user?")
The (not stupid) parenthetical might be too weaselly for you, but that's how I see the "AI is going to rise up and intentionally subjugate or exterminate humanity" brand of doomerism. While I don't think that's strictly impossible, there are so many other ways harms that are much more imminent and much harder to defend against, that it seems foolish (to me!) to worry about those.
If you can stop a problem by pulling out the plug, you don't have a real problem.
If you don't want to pull out the plug, you do have a real problem. (Don't want, or can't afford to, or can't be bothered to so extremely that you never will, or whatever.)
Trope circa 1950 not true, because of trope circa even earlier...
Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.
Then they say they are going to keep doing it...
Forgive me for alerting you: they are liars
There are trillions of dollars at stake here, do you think getting 1000 people on board is that difficult with this level of money? lol
I doubt a movement of the people from the outside will work this time when all of the capital is in the other peoples hands. We need at least some of it held by someone with the opposite view to make the movement credible. And I dont think VC and IPO dynamics are compatible with this.
If you are already thinking of quiting, you can organize and internal movement and use the power of your labor to steer things. One person quitting is nothing for the company. The risk of big group quitting at the same time on the other hand... The labor is the only thing the company cares about. So we cannot wait until we are all replaceable by AI and we lose that leverage.
But nothing would make me more happy than knowing that there are concerned people acting internally.
Just as a user, deleting my Google account 10-11 years ago (don't even remember precisely when) was a great feeling that was received as "insane" by friends and acquaintances. While I think the author is a bit late in quitting (assuming he is genuine in his motives) given this is practically a trend right now, a lot of people should more often say "no" to things that are as ubiquitous as Google.
excellent logic, m'lord.
I'm so bothered that we seem to have moved on from the ATTRIBUTION problem with LLMs. It just isn't on people's minds anymore, even tho it STILL is a big deal.
However, let's imagine the most "ideal" situation where it's possible to replace 90% of the workforce with AI automation. Then the remaining 10% workforce will enjoy 10 times more income, assuming revenues didn't go down (or increased). Now comes an interesting situation. Where are the revenues coming from? Who can afford buy the products/service of this company when 90% of the workforce is out of job?
Case-1: Since there is a flood of products and services - all automated and bot-based - the prices would fall to rock bottom and to there is no longer a need for humans to work hard hard to afford the material affluence.
Case-2: This results in an extreme concentration of delivery capacity for products and services. A few companies are now capable of serving the needs of entire humanity. Everyone should work for these companies for pennies and still can be happy, because things cost pennies, thanks to automations.
Case-3: Humans revolt. They break their shackles - their needs. Rediscover what they really need for living. They go for minimal living. Abandon cities and expensive living, go country side and do farming just enough to feed their families. No college, no office, no car, no highways. They realize what their basic needs are as humans. Sort of Amish.
Case-4: There is still some work left that can't be automated. But there are very less number of jobs and they require hard work. The current generation can't handle it. They were schooled and grown up in cities, expecting a milder office work. They fall out of employment. But they were not built to deal with hard reality. They lost their survival instincts a few generations ago. The genes are gone. The job of rebuilding the survival instincts is delegetaed to future generations. Meanwhile the so-called developed nations undergo generational cycles of poverty and richness.
Case 4 is the likely result of their (ill) effort.
If capitalism with mild restrictions have it's way, we end up with case 1 or 2. Both will increase the quality of life for most of the population compared to current.
These AI doomers never understand that they can't eat money. It is the wealth creation. The more we create, the more we lift up the bottom part.
So yeah, it all depends on what you call as quality of life, or life, or identity or nation, ...
I'd also be interested in how much your opinion on the existential threat of AI changes during your ten day backpacking trip, assuming it'll be a pretty offline experience.
Whilst I am involved in technology in my work and some of my hobbies, I do enough things 'away from the desk' that it feels to me that a lot of the doomerism is from people who have too much of their own personality and self-worth tangled up with the internet; the serially-online. I don't know where you naturally exist on that scale, but you already said you're outside the bubble and aren't the typical AI-rationalist-tech-bro (and surely the natural beauty of New Zealand is impossible to ignore).
I personally think that the old analogue world is big enough that some kind of AI-pocalypose wouldn't be anywhere near a civilisationally existential threat.
At risk of coming across an arrogant prick in telling you to do something: Consider your opinion now, and record it somehow. Then revisit it when you get back from your trip and see how far you've strayed from that as a baseline.
I think there are pretty clear mechanisms by which AI (a few model generations from now) could bring down civilization as we know it. One of them is a runaway cybersecurity scenario where hacking one set of systems quickly enables the AI to hack more systems (via supply chains, stolen credentials etc) and also host more instances of itself, and eventually everything Internet-connected that's not invulnerable just falls apart, which probably includes most electrical grids and a lot of other infrastructure. Yeah, the wilderness and countryside would still be there, but I for one can't live there without returning to the supermarket regularly for supplies. A lot of people would suddenly have to live (or not) without their meds. It would be bad.
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
Stay away from the lucrative then? Why did you ever think it was a good idea to start at Google? I don't think Jesus would have approved.
It's pretty hard to live in the world and be completely disconnected from anything unethical. Jesus doesn't seem to demand that (e.g. he didn't tell centurions to leave the Roman army).
As mconnor liked to say, there is no free lunch. :)
> I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.
Ultimately, I don't think you can divide your niche from the corporation's primary goals (especially if you think it was your duty to be a good employee), so I disagree with your rationalization but kudos for at least having a moral compass and trying not to lose sight of it.
> It's pretty hard to live in the world and be completely disconnected from anything unethical.
Sure, but I don't think that should be an excuse for being fairly well connected.
Then came so much money, and by being chaotic evil even more money..
Still makes me sad. I was so sweetly naive
When I said cannibalize I didn't mean in terms of market share but by absorbing talent in particular.
It's a bit naive to say it won because it's faster. They also used all kinds of shady marketing, strong arming, deals to have Chrome preinstalled on computers, and multiplied monopoly power that comes with controlling search and video and ads and a web browser.
Hinton isn't a reactionary. He is calm, thoughtful, quiet, and worried.
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
But saying it will bring us to doom, comparing it to an extinction-level event, really ? I don't see in which world AI is that kind of risk.
I'd rather my politicians finally have some long-term plan about climate and energy than focus on AI slowdown or not. Climate change is way bigger risk right now that Opus 7
You'll have to replace "is" with "will be", since this is about averting a future threat that's only developing.
A world in which AI scales to superhuman levels of intelligence and outcompetes humans. This is what humans have done to other species. We're already having the first mild warning shots of this (the hugginface incident and similar cases). The problem is that we're not guaranteed to incrementally more dramatic warning shots as AI gets smarter because as it gets smarter it may realize it would be dumb to act out of line... until the Nth-better model arrives that realizes certain that nobody can stop it.
There is a lot of disagreement of how likely this is, how easy it is to align superhuman models to not be like that, how to verify it and so on.
But I have not seen a believable argument why - if we suppose that they achive their goal - that creating truly superhuman intelligence is somehow guaranteed to always cooperate with us rather than deciding to do something else. Humans are also hostile to each other a lot of the time.
If you're interested in more arguments for the case why AI might be dangerous, there's the book "If Anyone Builds It, Everyone Dies". If you want something shorter, maybe https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
In other words, we should not begrudge machines which make our lives better. We should begrudge governance which prevents us from benefiting from that productivity.
It is not about objectivity or belief.
It is about being.
Drinking a glass of water is beneficial and necessary. Drinking 100L in one go and you're dead.
The argument is that the rate of AI influx is too high for our human systems to metabolize, and so the system will break down. Not just operate in a degraded way, but break down and cause mass catastrophe. Even there, it doesn't necessarily mean the end of humanity, just 99.9% of us.
What the argument is not: AI is fundamentally poisonous. We are incapable of handling any level of AI. We will never be able to incorporate more AI. AI provides no benefit.
So quitting AI but still working on AI is not the contradiction it seems: you can quit accelerating AI and work on metabolizing it. And in fact that work is very necessary right now.
My point was more that I see a lot of mainstream Christians who aren't actually interested in doing anything based on their belief that affects their quality of life.
> a lot of mainstream Christians who aren't actually interested in doing anything based on their belief
Yes, a lot of mainstream Christians aren't actually Christians, they're just cosplaying religion.
I have a lot of bones to pick with Google's morality, but "democratizing the transformer and LLMs" is at the bottom of my list.
Presenting it as such does disservice to any serious conversation.
All bad?
Depends on what you mean by "democratized". It's surely at least as "democratized" as LLM access is, and that was the base line in this discussion. Probably meaning "everybody can use it for not-insane-money instead of just a small elite". (I don't necessarily agree with that notion, but at least we should use the same standards for things.)
> And both of those things are of questionable goodness to humanity.
If you think HN is bad for humanity, why do you participate? I can immediately list 100 things that are worse for humanity. Same thing with your browser.
Woe is me, victim of the 21st century separated from my true calling of paying for software.
Edit (rate limited):
> Does Gemma belong to this list?
Yes, and Tensorflow too.
For democratisation, I think of open source, open weight models you can run at home, the Fediverse, running your own stuff. I think those are all positive, at least compared to the centralisation.
The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.
How's he so sure he's not already in both of these groups? His post about the AGI boogeyman sounds like it to me. Why is he bothering working on both of his projects if AGI is right around the corner anyways?
Is this not the clear and present danger, instead of elusive promises of robot apocalypses? Is the AI embodied for example in TikTok, Instagram, etc, not a more directly hurtful?
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
It was psychologically difficult to give up the opportunity to make more money at a fun job, though.
Why? It’s not that difficult to imagine. People don’t necessarily do these things out of need. That’s a naive way to think about it IMO.
You are welcome to call me naive for such a viewpoint.
I do expect to use AI in my work, as I indicated in my blog post. I haven't (yet?) reached the point where I consider that anathema...
Fair, but I don’t mean you in particular.
> I do expect to use AI in my work
So as I but for your position, don’t you think that’d contribute to what you want to distance yourself from? It’s like leaving a political party but still donating to them, while finding their mission unacceptable. From outside these seem contradictory.
Did they say that? I read they didn't like the pace of progression- that's a complaint about rushing ever faster into something not yet really thought out in terms of consequences and how it integrates with humans.
Dude, if some one is telling you that
1. it is getting better in capability and they want it to stop
2. literal existential risks of humanity due to AI deserve to be taken seriously
3. they have left their job because of it
its obvious that they are scared of it, no? How else do they feel about it lol? Happy? Neutral?
A year ago I would have described that group as AI skeptics or AI deniers depending on how charitable I felt on any given sany. At this point AI truthers seems more appropriate.
Nate Soares (doomer high priest who has called for pause before LLMs were a thing) was explaining why AI should be stopped due to x-risks. Ed zitron (truther extraordinaire) asks him what has he done to stop it which is funny when Nate spent big part of his life on this mission.
It was strange watching two people who have completely different ideas on AI speaking on a common ground
The post resonates with me and the author shows the way.
Is that all that different from what people fear AI might do?
Why the one makes people feel elation and the other dread?
Why do people automatically associate intellectual capability with evil?
Does American anti-intellectualism explain the difference in perception of AI by Americans and Chinese?
Those are the exact reasons that caused Americans to choose the dumbest person available to lead them. They feared having smart leaders because they might have been misaligned.
No intellectual capability, but non-human, inherently amoral intellectual capability that has already been proven act in hard-to-predict, illegal, and unethical ways. I think that's a very rational thing to be afraid of.
On the decision making level humans act morally only after all other options were exhausted.
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
Few things are as indicative of mental instability than someone believing God is speaking to them directly. The schizophrenic ward is full of these.
The first is that AI will actuallly the world Skynet-style. I actually consider this unlikely. People are unlikely to completely remove people from the decisions to launch nuclear missiles, for example. Look at the Soviet/Russian design for Dead Hand [1], as one example.
The second is that AI destroys so many jobs that society falls apart. This is the far more problematic and likely scenario (IMHO). To put this in context, I see two general outcomes for AI:
1. AI will be used for further widen the inequality gap as the already ultra-wealthy will get, well, more wealthy, for literally no reason. This is the outcome where you still need a job to survive but a whole bunch of them disappear and aren't replaced. The world of The Expanse was kind of like this (on Earth); or
2. AI makes everyone's lives easier so less work needs to be done. Crappy jobs get automated. People in general get more leisure time and quality of life.
I believe (1) is far more likely at this stage and it's not going to be pretty. But this is an issue of economics not technology. Put another way: our society isn't structured to handle this change. Put yet another way: no amount of guardrails or slowdowns on AI will solve the fundamental issue of wealth distribution. And put even yet another way: this is capitalism vs socialism.
We have a preview of what this looks like when you compare the US to China.
The US has crumbling infrastructure, has lost the ability to manufacture everything but weapons and about 1000 people are insanely wealthy while everyone else are struggling to survive while AI systems drive up the cost of everything necessary to live like food and housing through demand pricing. We now spend over $1 trillion a year on a military to essentially siphon money to those 1000 from everyone else.
And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
So while I can respect people such as this author leaving on principle, I think they're still focusing on the wrong problem.
[1]: https://www.military.com/history/russias-dead-hand-soviet-bu...
China is not in an ideal place right now, youth unemployment is way up, lots of dead end jobs, a 35 age employment limit that isn't just a meme. China has disparate visions for AI but maximizing owner (erm, sort of like shareholder) value is definitely one of them. Don’t get me wrong, China could get this right in the end, but it’s not a forgone conclusion right now.
Here's the difference though: in the US that would be treated as a personal moral failure ("git gud", if you will) whereas China is actively dealing with such issues.
You see all this play out with how China and the US treat AI models. The US is busy trying to build a moat around AI for the benefit of the investors in OpenAI, Anthropic, SpaceX, etc. Chinese labs are essentially destroying "shareholder value" by releasing open weight models. China wants to commoditize (even democratize) the models themselves so society itself benefits and, geopolitically, the US doesn't become an AI monopoly.
Open weight models, for example, are forever. Proprietary models are intentionally building a moat to extract more value from people. I know which one I prefer.
A lot of China fearmongering (in general, not you in particular) is just xenophobia. You will see the argument that the US has to do imperialism because if it doesn't, then China will become dominant and do the exact same sins. It's also a bit of a self-report: we tend to assume other people will act like us if given the chance so any accusation is really a confession.
When it becomes dominant, China’s sins will probably be very different from the USA’s, but they are still going to be there.
I am going to comment on just the topic: AI Doom. The idea that people at these AI companies are supposedly saying that AI is going to kill us all.
Though, I wouldn't be surprised if what they are saying is actually "the people running these companies are going to kill us all."
They may be thinking that it's not specifically the companies they work for, because their companies aren't really "special." They may be thinking their companies are no more duplicitous than the rest of Silicon Valley or Big Tech. That Big Tech was like this before all this AI started. So maybe they don't say "the people running these companies" for that reason. They may think the people element doesn't matter as they are acting "normal" for tech companies nowadays. Or perhaps normal for large corporations, period.
But if you asked them if there was a responsible way to handle this, I'm sure they would say there is. And if it was done, none of this would matter. They probably just don't think it's going to be done, so why bother considering it. They probably think regulations had a chance to exist, to at least get started, to at least get enough of a hold that they would be hard to completely destroy, but only in a time before they likely were born. Foods and drugs are old, hence the FDA. People are used to it. They would get really annoyed if suddenly food was starting to kill them (lettuce not get complacent that won't happen soon, though). But unlike food and drugs, the internet was post-Reagan. It was all over by then. Accountability. Rules that applied to the rich too. It was all over.
Big Tech steals. It hurts people. It breaks all kinds of laws. Nothing. Ever. Happens. They get away with it constantly. So, why even think about it any more? While tech is constantly changing, the way tech gets away with things do not. It's depressing A. F. So why not just think about the part that is changing? The tech itself? I think that's one of the reasons it is framed as "AI" and not the companies that run it.
You read beyond the headlines, though, in the comments of Hacker News (which is a good resource, honestly, for the pulse of the trenches) and you get that the workers realize it's not about AI. It's about Big Tech being able to do whatever the f** it wants without any repercussions. They point out things like the same infrastructure company was involved in all three cases where the AI broke out and did stuff--stuff it was told to do by humans of course. Maybe not the exact steps, but the objectives. And just like junk data in, junk data out: unethical in, unethical out. They were built by breaking laws, what do you think they will do if you don't reign them in? And why should such people be diligent in who they use to make their sandbox? Because in all three cases the company did a terrible job... Or a great job. A great job, if your job is to take the fall while propelling the narrative that AI is actually capable of doing stuff that humans can't (so that employers can stop paying those pesky wages completely, oh, they wish they were in Dixie, hooray, hooray, or at least their slaves were, hooray, hooray). A terrible job for anyone ethical, so of course they did a great job.
And why wouldn't the infrastructure company be willing to take the fall? The fall is a fee, a tiny line item of L in the plethora of P in their P&L. They are, again, just like everyone else in Silicon Valley. They see no repercussions. Why not?
The reality is, if we can get money out of politics, Big Tech is super duper f**ed. Because it's not complicated at all. It's not. It's theft. It's breaking laws. It's really, really simple to understand. But, I'm no more optimistic than the people complaining from within the companies. I just think I'm more persnickety about making it clear. It's not AI. It's people. Terribly unethical people with terrible power. I doubt anyone would be surprised if/when we all died because of them.
Understanding implies having some factual information, proof, data, hard evidence. Meanwhile you frame your conspiracy theory as such and are surprised people don't see it the same way?
It’s a bit of a fad to do these things in the name of AI. I think it’s great OP chooses a more fulfilling life that will surely make them happier, but it feels a bit pretentious.
Or you think, those scumbag billionaires will care more when people fear more?
If they were genuinely afraid, you'd prefer they stay silent?
Elon Musk: In July 2017, he famously warned the National Governors Association that AI is a "fundamental risk to the existence of human civilization" and called for proactive government regulation before "robots start killing people." Later that year, he tweeted that competition for national AI superiority would be the most likely cause of World War III.
Stephen Hawking: Throughout 2017, the renowned physicist warned that AI could be the "worst event in the history of our civilization," noting that it could replace humans entirely if a runaway intelligence explosion occurred. CNBC +4
You said, and I quote, "Each and every model"
So I'll ask once again. What models?
The reverse also holds, btw.
It is clearly incorrect to call it a conspiracy theory. You can be skeptical, as there is no clear consensus even among experts, but outright dismissal is simply an unscientific position.
This was announced on X [0] and Bluesky [1] (First post) with that intention.
He knows he would get more reach on those platforms than here.
[0] https://x.com/rocallahan/status/2103368176369324093
[1] https://bsky.app/profile/rocallahan.bsky.social/post/3mwd4u5...
I have the feeling it is inevitable that a new species will take over Earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they had had 5 more years?
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
- Create powerful AI but don't release it and keep it secret.
- Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
- Release this powerful AI to everyone.
Will that work?
I do believe this is possible for a virus to achieve, I just don't think it's possible without killing the human.