Manifund foxManifund
Home
Login
About
People
Orgs
Categories
Newsletter
HomeAboutPeopleCategoriesLoginCreate
🦁
🦁
Serhat

@atilgandev

25

$10.1total balance
$10.1charity balance
$0cash balance

$0 in pending offers

Projects

AURA: Metric 3D facial reconstruction from phone photos
Evidince admission for computational research
By Serhat17Following12
AISA Table top exercises for AI Governance
🦁

Serhat

about 1 month ago

I went through the proposal, and I'm struggling to see what the 258k is actually expected to achieve beyond another round of high-level meetings.

Tabletop exercises can be useful, but where is the evaluation methodology? How are you measuring preparedness before and after the exercises? Who independently assesses the decisions participants make, and how do you verify that identified weaknesses actually get fixed?

Building relationships, improving understanding, and forming working groups are activities. They're not evidence that governments have become better at handling AI incidents.

You also received 100k for the India AI Impact Summit, where you set specific targets: 3–4 jointly developed proposals and at least one with senior decision-maker support. Were those targets met? Did anything translate into an implemented protocol, funded pilot, or operational change?

The budget is another concern. For a $258k request, listing staff, events, and administration without a detailed breakdown makes it difficult to assess cost-effectiveness.

And the failure section mainly mentions unavailable officials and political changes. Those are external obstacles, not criteria for determining whether your intervention actually works.

Frankly, I'm also disappointed that this proposal has attracted funding without these methodological questions being publicly answered. And I think the funders deserve scrutiny here too. Did you actually examine the methodology and evidence before committing money? Did you ask for independent evaluation, measurable outcomes, or proof that previous exercises accomplished anything beyond producing reports and meetings?

Funding AI safety research should require more than plausible narratives and institutional credibility. Where is the scientific rigor?

My question is: What concrete, independently verifiable result would demonstrate that these exercises actually improve AI crisis preparedness? And what result would convince you that the approach isn't working?

Without clear answers, I don't see how we're supposed to distinguish a successful AI safety intervention from a well-organized conference.

Forethought
🦁

Serhat

about 1 month ago

I went through several of Forethought's papers, and I have some serious concerns about the gap between the strength of the conclusions and the evidence supporting them.

Take the software intelligence explosion paper. The estimated R&D returns parameter for computer vision ranges from 0.8 to 2.4, crossing the very threshold that determines whether the model predicts an explosion. The analysis also relies on researcher headcounts as a proxy for actual research effort and extrapolates historical efficiency trends into a fundamentally different future.

That's an interesting hypothesis, but how much predictive confidence can we reasonably extract from it?

The AI-enabled coups paper has a similar problem. It identifies plausible threats, but where is the evidence establishing their relative likelihood? And how would we experimentally evaluate whether the proposed safeguards actually reduce those risks?

My broader concern is how research impact is being measured here. Publications, citations, podcast views, and influence within the same research community aren't necessarily evidence that real-world risks have been reduced.

I'm not questioning the value of theoretical research. I'm questioning the standards of evidence being applied to it.

Given the millions raised, I'd genuinely like to know: what concrete, independently verifiable outcomes would demonstrate that Forethought's research is working? And what findings would make you conclude that a major part of your research agenda was wrong?

I think those are reasonable questions when research is being funded to influence decisions with potentially enormous consequences.

Falcon Fund
🦁

Serhat

about 1 month ago

@Jasmine Thanks Jasmine. I read HarvestBench, and I like that the outcomes come directly from the game logs rather than an llm judge. One thing I couldn't find was a separate rate for cases where the model explicitly acknowledges the welfare concern but still chooses to continue. Have you considered measuring that mismatch across the existing transcripts? It could help distinguish models that never raise the concern from those that recognize it but override it, without changing the prompts and potentially affecting their behavior.

*Also noticed the provider-dependent shifts in Appendix C. That's a pretty striking reproducibility issue.

Falcon Fund
🦁

Serhat

about 1 month ago

@Jasmine Thanks Jasmine I had a look at TAC. That's very close to what Iwas getting at, especially the focus on actual decisions rather than just what models say.

One thing I'm curious about: have you looked at cases where a model notices the welfare issue on its own but still goes ahead with the harmful choice?

That gap between recognizing a concern and actually acting on it seems worth measuring separately.

The Metascience Observatory Replications Database
🦁

Serhat

about 1 month ago

I wonder how you'll distinguish a failed replication from two studies that weren't quite testing the same claim.

In medical imaging, for example, changing the alignment method or excluding difficult regions can substantially change the reported accuracy, even when the underlying system hasn't changed. Preserving those evaluation choices alongside the replication outcome could be really valuable. Otherwise, a simple success/failure label might hide the most interesting part of the disagreement.

Swarm incidents tracking platform
🦁

Serhat

about 1 month ago

One thing I'd be careful about is confusing an increase in incidents with an increase in reporting. A single swarm incident could generate dozens of articles and posts, while another might barely get noticed. It might be worth assigning each incident a stable ID and keeping the original evidence separate from later reporting. Otherwise, the dashboard could end up measuring media attention rather than actual swarm activity.

AURA: Metric 3D facial reconstruction from phone photos
🦁

Serhat

about 1 month ago

A small update on the validation side.

I've been in touch with a professor at the University of Birmingham who has experience with 3dMD facial imaging. He suggested that a new prospective study would be the better approach and mentioned the possibility of involving a student.

Nothing has been formally agreed yet, but it's an encouraging step. The next task is to work out a practical study protocol using paired phone photos and professional 3D scans.

I'll share more as things progress.

Research into relative effects of different pacing interventions
🦁

Serhat

about 1 month ago

I like the idea of an incidence column, but I'm not convinced its sign tells us whether a regulation is genuinely about safety or just protecting incumbents. A rule could reduce real risks while also benefiting the leading lab, simply because smaller competitors are less equipped to comply. Both things can be true. Wouldn't it make more sense to measure those effects separately? Curious what gleech thinks about this.

Falcon Fund
🦁

Serhat

about 1 month ago

Something I'd be curious about with the midtraining project: can a model learn to care about animal welfare without being explicitly prompted to?

It's easy enough to teach a model to give thoughtful answers about animal suffering. The interesting test is whether it notices the consequences for animals when making an otherwise ordinary decision, especially when doing so conflicts with cost or efficiency.

That seems like a better way to tell whether the training changed its priorities or just its vocabulary.

Video Game Eval Compilation and Analysis
🦁

Serhat

about 1 month ago

I think the real value here is in identifying failure modes rather than ranking models by game completion rates.

Gavin's Pokémon results illustrate this well: a model can appear competent until a small change in the environment exposes how brittle its strategy actually is.

What I'd find particularly interesting is whether those same failure modes recur across very different games. That could tell us something about the underlying limits of generalization, rather than just how well a model has adapted to a particular benchmark.

Living support for Pip Foweraker
🦁

Serhat

about 1 month ago

What stands out to me is that Beacon had $250k in funding lined up, yet Pip still had to cover expenses personally while the paperwork dragged on.

@Austin, have you seen a good way for fiscal sponsors to handle this kind of gap? Seems like something that could happen to a lot of early-stage nonprofits.

AI Safety Workshops For 6th-8th Graders in India
🦁

Serhat

about 1 month ago

@NeelNanda I think the key issue is that the proposal conflates three different outcomes: expressing interest in AI safety, actually entering the field, and becoming a researcher at your level.

There's also a numerical mismatch. If 1% of 1,350 students pursue AI safety, that's 13.5 students, making the cost roughly $11.1k per prospective entrant, not $9,357 per researcher.

@aashkapatel, do you have any follow-up data from Phase 0 on actual participation or completion, beyond initial interest? That would make the Phase 1 projections much easier to evaluate.

Doom Debates — Serious AI X-Risk Debates & Analysis as Mainstream Infotainment
🦁

Serhat

about 1 month ago

Liron, one idea that might make Doom Debates even more valuable: what if every debate ended with each side stating exactly what evidence would change their mind?

You could publish those answers alongside the episode and revisit them later. It would give viewers a way to see whether the disagreement is actually being resolved over time, rather than just debated.

Have you considered trying something like this?

AURA: Metric 3D facial reconstruction from phone photos
🦁

Serhat

about 1 month ago

@adeviye38 It’s a mix. ML is used on the landmark and face-model side of the pipeline, but Aura isn’t just asking a model to generate a plausible 3D face. The reconstruction is constrained by what the different photos actually agree on — things like multi-view geometry, depth, facial contours and metric scale.

The prior helps where the image evidence is weaker, but I don’t want geometry coming from a model to be presented as if it was directly measured. Aura keeps track of that distinction. If a reconstruction doesn’t pass the geometry and evidence checks, it can be rejected instead of just being shown anyway.

I also deliberately don’t treat generative completion of unseen parts of the face as a measurement. If the system can’t really observe a region, I’d rather have it say that than invent something that just looks convincing.

In our current internal 10-subject benchmark, the reconstruction pipeline achieved 0.29 mm median error against 3D reference scans. The next step is independent validation on a much larger dataset.

So the goal isn’t just to make a realistic-looking 3D face. It’s to make one that can tell you what came from the photos, what came from the model, and when it shouldn’t claim to know something.

AURA: Metric 3D facial reconstruction from phone photos
🦁

Serhat

about 1 month ago

@carol Thanks for flagging this. I understand the issue, the activity has stopped, and it won’t happen again. I’d appreciate it if the moderation note could be removed or updated to reflect that the issue has been resolved

AURA: Metric 3D facial reconstruction from phone photos
🦁

Serhat

about 1 month ago

@sibelzeynepumay Thank you very much. The key now is proving the accuracy independently.

AURA: Metric 3D facial reconstruction from phone photos
🦁

Serhat

about 1 month ago

@fattmaatilgan Thank you. That access gap is exactly what we’re trying to close.

Transactions

ForDateTypeAmount
Manifund Bankabout 1 month agodeposit+10