Search Explore

  • ekinertac · 4 days ago

    Author here.

    To be upfront about what this is: I'm not a Rust developer or a PHP internals person. This is an experiment in whether the "point the AI at the original project's test suite" methodology (the way Bun was driven against real-world suites) holds up when the human can't review the code. The oracle is php-src's own .phpt corpus, ~22k tests I didn't write. Current honest score: 3,844 passing (17.4%), with a realistic ceiling around 40-45% since the rest tests C extensions (GD, curl, intl, etc.) that are out of scope.

    "Renders WordPress" means: fresh install completes into SQLite, the front page renders with real posts, a real theme and /wp-admin/ renders without issues. The REST API is untested, and it's currently ~55x slower than PHP on the front page (a bytecode VM is in progress, micro-benchmarks are already at 1-3x of PHP 8.5).

    The scoreboard auto-generates into the repo after every run, whether the number went up or down.

    Happy to answer anything.

    • bbg2401 · 4 days ago

      Will you answer questions yourself, or will you simply pass on what your LLM of choice writes for you?

      Edit: On further inspection, the blog design, the blog build, the blog articles and even the anecdotes used in the articles are entirely Claude generated.

      Stop being so lazy. Get Claude to do something interesting and use your own intellect to assess and challenge the work in your write up. Or the other way around. Inject some amount of human work, at least. Otherwise, what's the point in sharing?

      • ShinyLeftPad · 4 days ago

        > will you simply pass on what your LLM of choice writes for you?

        But it will be as least 17% correct!

        • cataphract · 4 days ago

          The "honest score" is the most annoying claudism of the comment, with the short disjoint sentences a close second.

          • superdisk · 4 days ago

            It was "I need you to sit with:" that immediately made me close the article. I like LLM programming, but I really don't understand why so many people just post LLM-generated articles. What did the human even do at that point, press the start button?

          • ekinertac · 4 days ago

            well, thanks for the tips on how to run my own blog :) but the post already tells you this, the last paragraph literally says an LLM drafted it and i edited it. the whole project is an experiment in what a non rust/php guy plus AI can ship, so hiding the AI in the writing while disclosing it in every commit would be a weird place to draw the line.

          • adamtaylor_13 · 4 days ago

            This is a pretty cool experiment. Thanks for sharing!

            • ekinertac · 4 days ago

              thanks! the devlog in the repo has the longer war stories if you enjoy this kind of thing.

            • pluc · 4 days ago

              Compare with FrankenPHP?

              • ekinertac · 4 days ago

                I'm not there yet but I'll run the benchmarks against FrankenPHP and include it in the project when we get at least %60 test parity.

            • AmazingEveryDay · 4 days ago

              Interesting read. Given what the process is producing it's probably quite cost-effective?

              • Chaosvex · 4 days ago

                What do you mean? What's cost effective about this?

              • keepupnow · 4 days ago

                Why stop at 17%, come back when you are at 100% otherwise it's just another project.

                • lawrenceduk · 4 days ago

                  Is it astonishing you got to 17% with some vibe code? Sure.

                  But most of the stuff I’ve vibe coded this year has been astonishing by 2025’s standards.

                  If you got 100% I’d be genuinely blown away.

                  • sdesol · 4 days ago

                    The article doesn't go into how they managed the AI context when implementing things but I would not be surprised if it was done in a methodical way, 80% - 90% of the test could have passed.

                    • pylua · 4 days ago

                      Does anyone know why we write code anymore? Why not pass through to an llm that generates the page on the fly (ssr)?

                      Is it cost ?

                      • Jabrov · 4 days ago

                        Yes: cost, speed, and reliability.

                        But all of those things are improving at shocking speeds, so I think we’re on a path where code is losing value quickly.

                        • pylua · 4 days ago

                          Yeah, I agree. It will be like serverless but for code : codeless.

                          It’s a disconcerting future.

                          • block_dagger · 4 days ago

                            Why disconcerting?

                            • pylua · 3 days ago

                              Seems like the reshaping of an industry and is disconcerting for those of us who work in it.

                      • general_reveal · 4 days ago

                        Standards vary.

                        • UncleEntity · 4 days ago

                          What I suspect is this 17% is the exact sub-set it needed to hack together to make the goal (running some example website) a reality as this is what those dodgy weasels do if you let them. Then you get to spend 200x the time to fill in the rest of the "speculative features deferred due to no real consumer" on top of whatever dodgy system they made up, which is usually whatever is easiest/closest to the literature instead of the actual intended design. Lots and lots of fun to be had doing the full-pipeline refactors to add that last 2% which need support from tip to tail.

                          It's all in good fun, though... probably?

                        • gamblor956 · 4 days ago

                          Maybe the takeaway is that 20% is about all the LLM can muster.

                          • malisper · 4 days ago

                            > Maybe the takeaway is that 20% is about all the LLM can muster

                            At this point there's a long list of projects that have used LLMs to rewrite a system in Rust including:

                              - Bun (https://github.com/oven-sh/bun/pull/30412)
  - Valkey (https://github.com/ianm199/valdr)
  - Git (https://github.com/gitbutlerapp/grit)
  - Postgres (https://github.com/malisper/pgrust)
                            With the exception of Bun, these projects were done pre-fable too, so I bet Fable will make these types of rewrites even easier.

                            • verandaguy · 4 days ago

                              I'm not sure about the other three, but Bun's rewrite from Zig to Rust was a bit of a joke. `unsafe`s in the thousands, a quarter-million lines of diff, and merged inside a week with no significant public discourse (at least, not much that was responded to by the author).

                              • solid_fuel · 4 days ago

                                Still waiting on that blog post from Jarred that will supposedly answer all the questions and concerns about the rust port.

                              • gamblor956 · 4 days ago

                                I think the standard should be rewrites that are at least as good as the original, not buggy piles of unfinished unmaintainable crap.

                              • UncleEntity · 4 days ago

                                I mean, I got them to 100% using the official conformance suite on my copy-and-patch jit compiler/interpreter WASM VM...

                                Saw that Salt Language article a day to two ago on how they do the static verification as part of the compilation process (or whatever they really get up to) and that's next on the agenda, tried that with a JavaCard VM I was poking at as its 'computation space' is much smaller but that was too much for my poor little laptop to handle but, apparently, this Salt thing is much different and actually tractable so, we'll see, still working out the details.

                              • fuckinpuppers · 4 days ago

                                Use AI to make Wordpress secure and not suck as much

                                • lioeters · 4 days ago

                                  Even an AGI can't accomplish the impossible.

                                • sshine · 4 days ago

                                  My boss asked me to set up a WordPress for a product landing page.

                                  I naturally won't do this; it's no more than a couple of weeks ago that some SQL injection landed in the search query function of this monstrosity.

                                  WordPress always was and always will be terrible.

                                  So I set up the landing page with a Hugo static site, and I've been vibe-coding a WordPress-like dashboard that operates on git repositories containing Hugo sites.

                                  I call it WorbPress (not released yet), and I'm sure that's what my boss told me to install, or I might've misheard.

                                  And yes, it's written in Rust (with Axum and Alpine.js), because why not?

                                  • is_true · 4 days ago

                                    Why not use headless WordPress?

                                    • sureglymop · 4 days ago

                                      I feel like not choosing WordPress was a great choice but I'm not sure about the rest of the comment. A simple html file might make for a good landing page though.

                                      • brailsafe · 4 days ago

                                        > because why not?

                                        I'm not certain, but it seems like you're not being entirely serious here, however..

                                        If you aren't joking, or for other people in this position, I'd first wonder if the landing page required a search function that would hypothetically be subject to the vulnerability, then I'd wonder about what the normal nature of your business is and how much latitude you personally have in the allocation of billable hours to arbitrary technology choices and whether those do actually align with the deliverable, then if I was the boss I might wonder why you created a bunch of (potentially) out-of-scope random liability using unusual lesser-known tools based on a personal vendetta against WordPress.

                                        I've been in this position, conceptually if not literally, and I've probably been (in a way, rightfully) fired for it, but my country's labor protections are likely not quite as good as Denmark's.

                                        If there's a question about why money was spent on implementing a bunch of stuff nobody knows for a reason nobody cares about, especially for a very short-lived thing like a landing page, then it's a sticky situation if the answer is basically novelty. Something like this, if it does serve a purpose, should be planned for and a case made for it, but that also doesn't really seem like agency work.

                                        If I was asked for WordPress, which I have, and I delivered Rust, I don't think I'd keep that job, but mileage may vary.

                                        Most work is about solving problems as they are, not what we wish them to be, and if a 5 min job becomes a month long job that the customer didn't ask for, it's an extreme case of yak-shaving.

                                        • sshine · 4 days ago

                                          I get what’s you’re saying, and if I couldn’t justify making the best alternative I can imagine in my free time, because I’ve wanted it for a long time, I’d install “a CMS” (not WordPress).

                                          > If there's a question about why money was spent on implementing a bunch of stuff nobody knows for a reason nobody cares about, especially for a very short-lived thing like a landing page, then it's a sticky situation if the answer is basically novelty.

                                          The economy behind a decision like is this: alternative SaaS website builders are $20-60/mo./seat. We’ve historically paid $720/mo. for the ability to edit a single website that doesn’t look great but is dead simple to modify.

                                          So if I can make something that scales up to any amount of sites and any amount of editors with ~10 hours on landing the design (which isn’t included in “a WordPress” either way), at ~$700, then I can justify making ten sites per year at the cost of our first.

                                          Or more realistically: The total operating cost of the current website gives me 125 hours in a year to make something better.

                                          Then the question is not “Can I make something better?” (Yes.) Or “Is it affordable to make something from scratch?” (It is.) But rather: Could I make more money doing something else? (I could halve the Azure budget in less than a month by optimizing and cleaning up.)

                                          • brailsafe · 3 days ago

                                            Makes sense to me, your other comment that elaborated on the scenario adds some useful context. I assumed there was quite a lot missing and tried to frame the comment for future versions of myself that might think it's a good idea to just run with a wildly out-of-scope idea with too much hubris.

                                            • sshine · 3 days ago

                                              > might think it's a good idea to just run with a wildly out-of-scope idea with too much hubris

                                              I'll say that using git as a database instead of simply SQL leads to a lot of corner cases.

                                              For example, you are inside a transaction from the moment you make a single change to anything.

                                              But the filesystem is conventionally thought of as something that can be shared between sessions.

                                              So I need a git worktree per user.

                                              In both my use-cases there's very few users, but if there were two, and they decide to change things independent of one another, then

                                                - their worktrees will become stale, and I need to add somewhat automated checking how far behind they are
  - their stale worktree will have conflicts, and dealing with merge conflicts in a pedagogical way is a big task
  - working on multiple things at once is possible, but introducing "committed" on top of "draft" + "publish" adds mental gymnastics
  - using the filesystem as a database makes deployment to Kubernetes less trivial because you need persistent storage
                                              One thing you get "for free", though, is authentic preview: Since the final site is a static site, I can just run the generator when the user wants to see a preview.

                                              The nice thing about this is that if you remove the admin dashboard, it's just another static site.

                                              It can be maintained perfectly fine with regular git and e.g. VSCode.

                                              Add Nix + Claude and you have an entirely different mode of operation.

                                              Which is what I like about plaintext (Markdown). It's not tied to any one program.

                                        • kmoser · 4 days ago

                                          Just to clarify: you think your vibecoded dashboard is more secure than WordPress? Not saying you're wrong, just wondering why you think you're right. Are you auditing the generated code, or is it a giant yolo?

                                          • lopatin · 4 days ago

                                            Auditing the generated code would defeat the purpose of reckless insubordination.

                                            • fastily · 4 days ago

                                              I’m reasonably certain GP is (humorously) trolling us

                                              • sshine · 4 days ago

                                                Thank you.

                                              • sshine · 4 days ago

                                                How do you hack a static HTML page?

                                                The point is that most WordPress pages don’t warrant the dynamic code execution on every page load.

                                                When you use a static site generator and make content creation convenient behind the scenes, you move the entire attack surface to, in my case, nginx, the load balancer, and OpenSSL.

                                                • kmoser · 4 days ago

                                                  You don't hack a static HTML page, you hack the (presumably public-facing) vibecoded admin panel that can update the static page.

                                                  • sshine · 3 days ago

                                                    I think "yolo" gets overloaded here to mean "not thinking".

                                                    I've described it as static HTML.

                                                    The admin dashboard isn't static HTML, but it also isn't public-facing.

                                                    The admin dashboard lives behind an oauth2-proxy authenticated via keycloak, and only accessible to the corporate wifi / vpn. The dashboard is really a completely separate application.

                                                    The probability that you can hack an admin panel because you know the public URL assumes either there's a "Login" button on the front page, or that the admin panel depends on the source code in the same way as the public-facing pages.

                                              • techscruggs · 4 days ago

                                                Let me make sure I am hearing you right. 1) The person you report to asked you to accomplish a discrete task 2) of standing up one of the most common websites on the planet 3) and your response was to begin building your own custom CMS?

                                                I know I am removing the train of thought that led you down this path, but is there anything I just said that is factually false?

                                                • asp_hornet · 4 days ago

                                                  Is this the “taste” I keep hearing people say they bring?

                                                  • igetspam · 4 days ago

                                                    AI has helped us all lose the plot because now we don’t just know better than everyone else, we can prove it by project managing our better versions of the same things.

                                                    What I find really great is that we’re only a prompt or two away from proper docs for these novel solutions but we still don’t make them and if we do, we definitely don’t read them first.

                                                    • sshine · 4 days ago

                                                      I’ll say “proper docs” has shifted for me for two reasons

                                                      I used to insist on commenting code richly, so I could better read it. But comments lie, while code is truth. Read the code, that’s what it does.

                                                      With AI, the cognitive overhead of getting a human-worded explanation of what’s true, is one prompt away and is never a stale leftover.

                                                      So the purpose of docs: Specs for implementing and getting an architectural overview, and API documentation for exploring the interface of something new.

                                                      What I find great is that people still don’t test their code when it became practically free to do so.

                                                    • sshine · 4 days ago

                                                      I’ll try to tell the story in a more responsible way: My boss asked me to install a WordPress, to which I advised against it; while it’s easy to set up, it doesn’t align with our tech stack (his main team won’t be able to support it easily, woohoo army of juniors!), and the convenience of a quick start is outweighed by having a thing that needs CVE patching when, guess what never got hacked: pure, static HTML.

                                                      Since my wife had asked me twice the same week to set up a website with a design mock she’d sent me, I thought: what’s holding me back in both cases from giving them a Claude Design’ed Hugo theme is that they need to edit Markdown on their filesystem and run terminal commands.

                                                      So I picked an item out of my infinite backlog, which was very well-defined: a web dashboard that acts as the equivalent of the WordPress admin page that lets you manage a Hugo static site, use a rich editor on top of Markdown, and commit to git instead of a database. I spent the better part of a weekend making this, with my wife as the customer, and when it got good enough, I presented it to my boss. He was happy with the choice, but mostly because of the vibed design, he ultimately didn’t care about the technology.

                                                      When someone wants “a WordPress” they’re asking for convenience of an easily updated website.

                                                      You don’t have to actually give them a WordPress.

                                                    • librasteve · 4 days ago

                                                      what, no HTMX?

                                                    • wsor4035 · 4 days ago

                                                      Ill preface my comment with saying: this might not be the best solution give the goal of your project to iteratively loop through and improve on the tests each round, and using deps would make that process longer/more complicated having to work potentially with another project.

                                                      .....however.....

                                                      mago, a static analyzer for php is written in rust and might be useful for gaining some "free" performance uplift: https://github.com/carthage-software/mago. iirc it splits out a far bit of its internals so they can be used by other projects (citation needed)

                                                      • ekinertac · 4 days ago

                                                        thanks, mago is a cool project. probably not as a dep tho, the parser isnt where the time goes (the 55x gap is all in evaluation, thats what the bytecode vm is for) and our parser is deliberatly tuned to match php's exact parse error messages, which is itself worth tests in the corpus. but using it as a second oracle to cross check my parser against theirs is actually a neat idea, same trick as the phpt suite but at the syntax layer.

                                                      • mgaunard · 4 days ago

                                                        Why is the AI only able to reach 17%?

                                                        Surely it can just keep iterating until it implements the full test suite?

                                                        • hoppp · 4 days ago

                                                          Money probably. This is a cash burn project.

                                                          • dzhiurgis · 4 days ago

                                                            I suspect it cost less than $100 using chinese models.

                                                            • ekinertac · 4 days ago

                                                              it's not that expensive actually. been working on this about a month with my 20x Max Claude subscription while I'm working on other projects as well.

                                                            • ekinertac · 4 days ago

                                                              its still iterating, 17% is just where the counter is today. three weeks ago it was at 10%, two weeks ago 13.8%. i didnt post this as a final result, i posted becuase wp-admin rendering surprised me.

                                                              but no, it cant reach 100%. around 55-60% of the suite tests C extensions, gd, curl, soap, intl, mysqli, ffi, sockets etc. passing those would mean writing all those extensions from scratch too (libcurl, ICU, an image library...) which is a completely different project. the realistic ceiling for a from scratch engine is around 40-45% and thats the number im climbing towards.

                                                              • mgaunard · 4 days ago

                                                                most of those are just bindings, trivial work.

                                                            • t1234s · 4 days ago

                                                              I feel like the future will be a git repository with text files and a markdown file describing how the site should look and how any endpoints needed for functionality should work and the AI will be the runtime for your site instead of wordpress.

                                                                  • flurpitude · 4 days ago

                                                                    Using an LLM as a live runtime would always be a wildly inefficient and less reliable way to perform routine, deterministic tasks compared to traditional code.

                                                                • rbbydotdev · 4 days ago

                                                                  I’d be curious for a similar experiment converting frankenphp to rust.

                                                                  https://frankenphp.dev/

                                                                  • tensegrist · 4 days ago

                                                                    > Here’s the part I need you to sit with

                                                                    no, i don't think i will

                                                                    • MichaelMoser123 · 4 days ago

                                                                      Wow. Now did you try to check the setup with something like Claude Fable? Will it find issues, what kind of issues? Another question: how many tokens did this effort cost? Did you learn new prompting tricks?