Search Explore

  • harmonic18374 · 56 minutes ago

    A company whose first demo was completely fraudulent announces that its model beats GPT-5.5, on its own benchmark? I’m gonna wait a little before I trust this.

    This whole company seems to optimize for raising money and impressing VCs. Lying about their products, ignoring consumer market to target enterprise, bragging about how they work their employees like slaves, and writing these posts full of intimidating technical jargon...

    • giancarlostoro · 51 minutes ago

      Would love to see these companies use benchmarks done by third parties.

      • anthonypasq · 40 minutes ago

        they are right there? it shows swe-bench multilingual and terminal bench

      • w4yai · 49 minutes ago

        What happened ?

        • SubiculumCode · 45 minutes ago

          Link for this?

            • andy99 · 30 minutes ago

              Is it just me or does all that* seem pretty tame by today’s standards? Not saying it’s right, but it barely raises eyebrows. Sounds like a pretty typical startup demo.

              * Based on the first comment in the link that claims to summarize the video.

          • oa335 · 37 minutes ago

            > "A company whose first demo was completely fraudulent"

            Could you expand on this?

            • haritha1313 · 26 minutes ago

              Their product has far evolved beyond this (of course with the large amount of money being poured into it) and is now used by a lot of traditional companies (banks etc). Also the SWE models were originally built by Windsurf and this seems to be on top of that (after acquisition) although the original SWE-1 models weren't that groundbreaking.

              • achandra03 · 14 minutes ago

                To be fair it does seem like most AI startups are now like this (particularly when it comes to constantly mentioning how hard they work and ignoring consumer markets).

                • parineum · 6 minutes ago

                  > it does seem like most AI startups are now like this

                  Remember when AGI was going to replace all jobs in 6 months? It's always been like that.

              • throwaw12 · 54 minutes ago

                Open source for the win!

                Imagine how far community might have pushed if 2 past versions of 'morally superior' Anthropic and 'completely Open AI' open sourced their models for the community to build on top of them

                • spott · 47 minutes ago

                  Is this open source? I can't find a link to download the weights.

                  • UncleOxidant · 35 minutes ago

                    It's based on an open weight model (Kimi 2.7) so shouldn't it also be open weight?

                    • andy99 · 27 minutes ago

                      There is no obligation to do that. I think the landscape would be very different now if one of the big labs had released an earlier “frontier” model under copyleft that requires sharing fine tunes. I hope it still happens.

                • llmslave · 53 minutes ago

                  These models are never as good, the benchmarks dont tell the full story

                  • SubiculumCode · 44 minutes ago

                    Funny, the cheerleading at HN for leading Chinese models, but a non Chinese lab (building on top of a Chinese model) gets dissed here.

                    • llmslave · 41 minutes ago

                      all the open source models are a waste of time relative to the bleeding edge from openai/anthropic

                      • pixel_popping · 35 minutes ago

                        Not true since a few months, genuinely try GLM 5.2 and Minimax M3, especially in adversarial/gating... as a general model, I can agree, but as a coding model, they are not bad, comparable to maybe Opus 4.5 in real usage which is quite impressive.

                        • llmslave · 22 minutes ago

                          yeah but why waste your time on these models, just use the one that gets the better results

                          • somenameforme · 17 minutes ago

                            I was going to respond until I saw your account name lol.

                            • 9183726518 · 16 minutes ago

                              Because you can get them from more trustworthy providers or with hardware encryption.

                              • llmslave · 6 minutes ago

                                i trust anthropic/openai with my data far more than some random startup.

                              • nicoburns · 13 minutes ago

                                I actively prefer GLM-5.2 for some tasks. For simple tasks the results are just as good as e.g. Opus, and it produces results significantly faster.

                            • wongarsu · 30 minutes ago

                              At work I wouldn't want to use anything else. Compared to my salary a Claude subscription (or two) is cheap

                              For hobby projects I've completely switched to DeepSeek v4 pro. I spend less than on a $10 Claude plan and am not subjected to quota limits (when I have time and motivation, the last thing I want is a 5 hour quota running out). And the difference in model performance is fine for those smaller projects, most of which will end up abandoned or in a state of "good enough" anyways

                              And for utility tasks, those 30b models are also great. I'm a big fan of gemma4

                              • llmslave · 24 minutes ago

                                ive just got better things to do with my life than fuss with an inferior model. its like why hire a dumb employee over a smart one

                            • sosodev · 35 minutes ago

                              It's almost as if HN users aren't all the same.

                              • mirekrusin · 16 minutes ago

                                It's simple: close weights = not welcome.

                              • haritha1313 · 22 minutes ago

                                The reality is most people building their own models and providing that alongside SOTA ones don't really care about how great these models are. They just prove that 'hey we are smart enough to build our own models so you can trust us instead of going with a single provider like Claude via Claude Code', also a cheap alternative for cost sensitive/free users - at least this was the case for Windsurf, not sure if Devin Desktop still has that tier. They just need to hillclimb the benchmarks and show something reasonable enough there.

                                • spate141 · 22 minutes ago

                                  Benchmarks are just vibes with error bars... wake me up when it survives a week on a real codebase without hallucinating a package that doesn't exist.

                                • achierius · 50 minutes ago

                                  I've always had mixed feelings about Cognition. Obviously they have some very, very smart people working there (I even know a few), and they do make real products. But at the same time, they've made suspicious marketing claims more than once and even been caught making outright fabricated ones; and while they certainly seem to have shaped up from that, I still find their claims to be in a sort of grey area where they seem to avoid unfavorable comparisons and lean on their own benchmarks. Certainly when I've tried their models they have not been nearly as useful as comparable versions of Claude, GLM, etc. -- though I haven't had a chance to try SWE-1.7 yet.

                                  • nibbleyou · 47 minutes ago

                                    Unrelated: what's the point of "*equal contribution"? Why would someone specify this

                                    • edot · 45 minutes ago

                                      Because papers are often referred to by the first author’s name, and often the first author is the primary researcher and therefore deserves the extra credit. When two or more primary authors are equally involved, they’ll often do a random ordering but annotate this so that no one thinks one did more than the others.

                                      • nibbleyou · 41 minutes ago

                                        Interesting. Thank you

                                    • yousif_123123 · 44 minutes ago

                                      We need more models that optimize for coding and that can be cheaper than frontier models, like what SWE 1.7 and composer 2.5 are trying to do. I don't think there's an effort to make something GLM-5.2 level but focused only on coding.

                                      • UncleOxidant · 31 minutes ago

                                        Qwen was doing something like this with their coder models. But alas, they seem not to be releasing those anymore. Last one was Qwen3-coder-next.

                                        • yousif_123123 · 18 minutes ago

                                          Its crazy that OpenAI and Anthropic themselves aren't doing that. No attempts at reducing inference cost for code as far as I know from them.

                                          • gsibble · 17 minutes ago

                                            I use this model. It's pretty good but not Opus 4.8 or Fable levels obviously. I'm really hoping we get more models like it (and better) soon. I run it locally and it's great that way.

                                            • UncleOxidant · 6 minutes ago

                                              Qwen3-coder-next is very usable. But I don't think it's as good as Qwen3.6-27B (though it does run faster on my hardware). It would be great if we could get a Qwen3.7-coder, but I'm not going to hold my breath.

                                        • hedgehog · 44 minutes ago

                                          Heads up to anyone else curious, I installed the Devin CLI and SWE-1.7 is not currently available there.

                                            • mirekrusin · 19 minutes ago

                                              Open weight models should have GPL-like license where it says if you train model on it, it needs to be open weight as well.

                                            • pants2 · 38 minutes ago

                                              Kinda funny that their "cost-vs-performance" chart looks the same as the one for Composer 2.5[1], except that it includes Composer 2.5 at a completely different spot.

                                              What are the chances that CursorBench ranks Cursor's model highest, and Cognition's bench ranks Cognition's model highest? Both are to be RL'd from Kimi as a base model, BTW.

                                              I'd posit that it's not deliberate deception, but for both companies their training data and benchmarks come from the same dataset (Devin/Cursor interaction logs) so they naturally overfit.

                                              1. https://cursor.com/blog/composer-2-5

                                                • petesergeant · 8 minutes ago

                                                  > they left out ... GLM 5.2

                                                  They did not.

                                              • fallinditch · 35 minutes ago

                                                I'm looking forward to trying this out. I've been using SWE 1.6 quite a lot for grunt work alongside Opus for higher level planning and tricky stuff - a good combo.

                                                As a (former) Windsurf user I'm pretty happy with the progress of the Cognition/Devin ecosystem after they took over Windsurf, now known as Devin Desktop.

                                                • ryandvm · 32 minutes ago

                                                  Okay, let's give software engineers a break for a bit and focus on obsoleting other high-linguistic context occupations.

                                                  • spate141 · 23 minutes ago

                                                    Feels like they discovers that if you build your own benchmark, you can win it

                                                    • londons_explore · 17 minutes ago

                                                      Pretty sure most benchmarks are being gamed by people training on the test set deliberately or accidentally anyway.