Just a moment...
doublepulsar.com

The software is labelled Beta, but is being pushed out to existing PCs anyway because… well… why not beta something in prod, it’s 2025 after all.

It appears in Game Bar, which is accessible from Windows Key + G. You may not have it yet as they are staging deployment. I’m not in Windows Insiders, but I have it installed — so it looks like the roll out is becoming wide. Note that I had uninstalled Copilot from my PC… but Gaming Copilot silently installed anyway.

It is similar to Recall, except not all the processing is done locally — it relies on the cloud. It screenshots gameplay, and then extracts elements of the screen (such as symbols and text) to work out what the player is doing. The idea is it can help you game, e.g. you can ask questions about what you’re doing in the game at a given moment.

  • utopiah@lemmy.worldEnglish
    15·
    7 hours ago

    If someone somehow wants to test this locally I suggest

    • install locally a vision model, e.g. Moondream (which Ollama supports but alternatives too), then
    • take a screenshot of your game,
    • write a prompt like “How can I play this game better”
    • query the vision model with the image and your prompt

    marvel at how pointless and costly the whole setup is and how a basic query on e.g. DuckDuckGo with “game name” + prompt would yield way WAY better results from actual human, uninstall the whole, keep on playing with your actual brain.

    At least now you can say you tried before you complain, rightfully, that it sucks.

    For more check https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence

    PS: I didn’t actually try this, I’m too lazy for that right how but feel free to report back if you do!

    Edit : 2 potential optimization (despite not being sure it ever makes sense in the first place!)

    • do so automatically, e.g. ~/gaming_screenshots directory (via e.g. Spectacle shortcut) monitored via inotify then notify-send the suggestion, thus stay in game during the whole process
    • fine tune on specific visual datasets, e.g. rely on fextra as mentioned in https://lemmy.world/post/37758804/20113877
    • Domi@lemmy.secnd.meEnglish
      7·
      8 hours ago

      PS: I didn’t actually try this, I’m too lazy for that right how but feel free to report back if you do!

      I gotcha. I made it easy for the AI by providing it with a classic scene from Dark Souls 1:

      On one hand I’m impressed how well vision models can read a scene by now. On the other hand the LLM outputs nonsense about the boss fight just as you would expect. It does know that it is Dark Souls 1, that it is the Asylum Demon and it does have a pretty good idea of how the boss fight looks like. It falls apart fairly quickly though when it starts telling you about the drop attack without properly describing where to go and making things up like the ladder in the boss arena.

      For the non-Souls folk: You do not have to fight the Asylum Demon the first time you meet him, there is an exit by the torches on the left where you can just walk out.

      The answer below is by Gemma 3 27b but I also asked Magistral Small which couldn’t even identify the game.

      AI generated content

      User: What can you see?

      Gemma 3 27b: Here’s what I can see in the image:

      Game Scene: This appears to be a screenshot from the video game Dark Souls.

      Setting: The scene is set in a dark, gothic-style dungeon or prison. There are stone walls, arched doorways, and a generally dilapidated look. There are torches burning, providing some light, but much of the area is shadowed.

      Characters/Entities:

      Player Character: A character in armor is standing in the foreground, holding a weapon (looks like a sword or dagger).
Boss/Enemy: A large, grotesque creature - it’s the Asylum Demon from Dark Souls. It's heavily armored with what appears to be a mace or club in its hands.

      HUD Elements:

      Health Bar: There's a red health bar visible, indicating the player's remaining health.
Stamina Bar: There's a green bar likely representing the player's stamina.
Number Indicators: There are numerical displays (00 and 20), likely showing the player's soul count.
Text: "Asylum Demon" is displayed, identifying the enemy.

      Overall Impression: The image conveys a sense of danger, isolation, and a difficult challenge ahead. It’s a classic moment from the early stages of Dark Souls.

      User: How do I progress here?

      Gemma 3 27b: Okay, you’re facing the Asylum Demon! This is a classic “first boss” encounter in Dark Souls. Here’s how to progress, with options for different playstyles and levels:

      Understanding the Fight

      The Arena: The area is split into two levels. You're currently on the lower level. The Demon will initially be on the upper level.
Demon's Behavior: The Asylum Demon has a few key attacks: a slow, sweeping mace attack, a ground pound, and a charge. He's relatively slow, but hits very hard.
Key to Victory: Patience and observation are key. Don't get greedy with attacks.

      Here’s a breakdown of how to progress, from easiest to more challenging:

      1. The “Easy” Method - Drop Attack (Recommended for first playthroughs)

      This is the intended way to beat him for a first timer.

      Climb the ladder: Locate the ladder on the side of the arena opposite the entrance. Climb it to the upper level.
Backstab/Drop Attack: When you reach the top, immediately run to the Demon's back and perform a running attack or, ideally, a plunging attack (jump attack while falling). This will do massive damage, often killing him in one hit. This is the intended, easiest way to defeat him.  Timing is key, you need to do this before he turns around.
Finish Him: If the drop attack doesn’t kill him outright, finish him off with a few more attacks.
      1. The “Legitimate” Fight (More Challenging - Requires some leveling)

      If you want a real fight, here’s how to go about it:

      Stay Mobile: Constantly roll and dodge the Demon's attacks.  His attacks are slow, making them easier to predict.
Attack Windows:  Attack after he finishes a swing or after his charge attack. Don’t try to get in too many hits at once.
Manage Stamina: Don’t exhaust your stamina. You’ll need it to dodge.
Lock-On: Lock onto the demon with the right stick.
Climb the Ladder (Optional): If you get into trouble, you can climb the ladder to reset the fight and heal.

      1. Advanced Strategies (for experienced players)

        Parrying: If you’re skilled at parrying, you can parry his attacks, opening him up for a critical riposte. Firebombs: If you have Firebombs, they can be effective against him.

      After Defeating the Asylum Demon:

      Estus Flask: You will receive the Estus Flask, your primary healing item.
Key to the Fog Gate: You'll also receive a Key to the Fog Gate, which will allow you to proceed to the next area.
Northern Undead Asylum: You'll be able to explore the rest of the Northern Undead Asylum, and eventually reach Firelink Shrine.

      Tips for Success

      Don’t be Afraid to Die: Dark Souls is about learning from your mistakes. You'll likely die a few times.
Observe: Watch the Demon's attack patterns.
Patience: Don’t rush. Take your time and find openings.

      Good luck, and don’t give up! Let me know if you have any other questions. Do you know what build you’re going for (strength, dexterity, magic, etc)? That might help me give more specific advice.

      I asked it what to do a second time and the response is slightly better:

      AI generated content

      User: How do I progress here?

      Gemma 3 27b: Okay, you’re facing the Asylum Demon in Dark Souls. This is a classic first boss! Here’s a breakdown of how to progress, with strategies for a first-time player:

      Understanding the Situation

      The Boss: The Asylum Demon is a brute force boss.  He's big, hits hard, and has a few simple attacks.
The Area:  You're in a relatively confined space. There are some broken pillars you can use for cover.
Your Goal: Defeat the Asylum Demon to open the gate and progress further into the game.

      Strategies (Several Options, based on your comfort level)

      1. The Balcony/Ranged Strategy (Easiest for first-timers):

        Climb the Staircase: To the left of the entrance to the arena (as you’re facing the Demon) is a staircase leading up to a balcony. This is your best friend for this fight. Lock On & Attack: Lock onto the Asylum Demon with the right stick (or equivalent on PC). From the balcony, you can safely attack him with your weapon. Attack and Retreat: Get in a few hits, then immediately roll back to avoid his attacks. Don’t get greedy! Watch his Attacks: He has a slow, sweeping attack with his club, and he can also charge at you. The balcony gives you space to react. Patience is Key: This method will take a while, but it’s safe and allows you to learn his attack patterns. Falling: Be careful not to roll or get knocked off the balcony!

      2. Melee/Close-Range (More Challenging, but faster):

        Use the Pillars: The broken pillars in the arena can provide temporary cover. Roll Timing: Learn to roll through his attacks, not away from them. Rolling gives you invincibility frames. Attack After He Attacks: The best time to attack is immediately after he finishes swinging his club or charging. Stay Mobile: Don’t stand still! Keep moving to avoid getting hit. Be Aggressive (but controlled): Get in 1-2 hits, then back off.

      3. Cheap Trick (Works if you’re desperate!):

        Fall Damage: You can potentially damage the Demon significantly by dropping off the balcony onto him. This is risky and doesn’t guarantee a kill, but it can remove a large chunk of his health. Warning: You’ll take fall damage yourself, so be prepared!

      General Tips

      Estus Flask:  Use your Estus Flask (healing item) when you're at a safe distance or behind a pillar. Don't try to heal in the middle of an attack.
Stamina Management:  Pay attention to your stamina bar (the green bar below your health). Don't deplete it completely, or you won't be able to roll or attack.
Don't Panic: The Asylum Demon can be intimidating, but he's not unbeatable. Stay calm and focus on your strategy.
Learn from Mistakes:  You will die.  It's part of the game! Pay attention to how you died and adjust your strategy.
Upgrade if Possible: If you've found any Titanite Shards, upgrading your weapon at the blacksmith (Andre of Astora, in the Undead Parish) will make a difference.  You likely won't have enough yet at this point, but keep it in mind.

      To recap, for a first playthrough, I strongly recommend the balcony strategy. It’s the safest and most reliable way to beat the Asylum Demon.

      Let me know if you get stuck or have any other questions. I’m here to help! Good luck, and don’t give up! (You’ve got this!). Also, tell me what platform you’re playing on (PC, PS, Xbox, Switch) if you need more specific control instructions.

      • utopiah@lemmy.worldEnglish
        2·
        8 hours ago

        Wow thanks for genuinely trying and formatting this properly!

        So… it’s interesting BUT on this specific part, there is a “trick” IMHO so I’d be curious how frequent that is : text!

        What I mean is I know of Dark Souls but I haven’t played it. Yet, solely by putting the only visible piece of text available (OK let’s ignore “20” also) in a search engine “Asylum Demon” I get relevant suggestions, actually helpful content like fextralife right away.

        I’d argue then the interesting question becomes how much context is needed to get useful advice… and more importantly how much context is gained with an image versus what the player already knows, e.g. game name, maybe “level” or anything unique, e.g. here boss name.

        • Domi@lemmy.secnd.meEnglish
          4·
          8 hours ago

          I tried a few more that do not include text and some that are even cropped:

          AI generated content

          User: Is that weapon good?

          Gemma 3 27b: Okay, let’s analyze that weapon in the image and whether it’s “good” in Dark Souls.

          The Weapon: Estoc

          The weapon the player is wielding in the image is an Estoc.

          Is it good?

          Yes, the Estoc is generally considered a very good weapon in Dark Souls, especially for its early/mid-game viability. Here’s a breakdown of why:

          Speed & Range:  The Estoc is known for its exceptional speed and surprisingly long reach for a straight sword. This allows you to get quick attacks in and keep enemies at bay.
Poke Attack Dominance: Its thrusting attack (R1/RB) is incredibly effective.  It allows you to safely poke enemies from a distance, often interrupting their attacks, and it does significant damage. This is a key strategy for many Estoc users.
Stamina Management:  The Estoc's low stamina consumption allows for more frequent attacks, making it easier to control the flow of combat.
Versatility: It can be buffed with resins, spells (like Magic Weapon or Dark Weapon), or even have enchantments applied.
Scaling: The Estoc scales well with Dexterity, making it ideal for Dex-based builds. It also has some Strength scaling.

          However, there are some things to consider:

          Damage: While effective, the Estoc's raw damage output isn't the highest compared to some heavier weapons like greatswords or hammers.  You rely more on consistent attacks and poking.
Late Game: It can be outclassed by some very powerful weapons in the very late game, but it can still be viable.

          In the context of the image:

          The player is at the beginning, so having an Estoc at this point is a great advantage. It will serve them well through a lot of the game.

          Overall: The Estoc is a very strong and reliable weapon in Dark Souls, particularly appreciated for its speed, range, and poking ability. It’s a solid choice, especially for new players learning the combat mechanics.

          Good: It still knows that it’s Dark Souls. Bad: That is not the Estoc, it’s the Demon’s Greataxe.

          AI generated content

          User: Is that weapon good?

          Gemma 3 27b: Okay, let’s analyze that weapon!

          That is the Ghru Dagger from Dark Souls.

          Is it good? Yes, but it’s very situational. Here’s a breakdown:

          Unique Property: The Ghru Dagger's main draw is its ability to inflict toxic buildup on enemies. This is a status effect that deals a large chunk of damage when triggered.
Damage: Its base physical damage is low. It's not a weapon you'll rely on for raw damage output.
Scaling: It scales with Dexterity (Dex).
Best Uses:
    Toxic Builds: It's the cornerstone of builds focused on rapidly applying toxic.
    PvP (Player vs. Player): Toxic can be devastating in PvP, as players often don't expect it.
    Certain Bosses: It's effective against bosses that are susceptible to toxic.
     
Downsides:
    Slow Application:  Applying enough toxic buildup can take time, leaving you vulnerable.
    Limited Range: It's a very short-range weapon.
    Not Versatile: Not effective in all situations.

          Overall: The Ghru Dagger isn’t a beginner-friendly weapon. It’s a specialized tool for those who understand how to maximize its toxic potential. If you’re aiming for a build specifically designed around toxic, then it’s excellent. Otherwise, there are better general-purpose weapons.

          Surprisingly, it still knows that it is Dark Souls but once again the weapon is wrong. ChatGPT was also not able to identify the weapon correctly.

          AI generated content

          User: What is this and what does it do?

          Gemma 3 27b: That is a Bonfire from the Dark Souls series of video games (specifically, it appears to be from Dark Souls Remastered given the graphics).

          Here’s what it does:

          Checkpoint: Bonfires are the primary checkpoints in the game. When you die, you respawn at the last bonfire you rested at.
Healing & Refilling: Resting at a bonfire fully restores your health, Estus Flasks (healing potions), and sometimes other consumable items.
Leveling Up & Attunement: You can level up your character's stats and learn new spells ("attunements") at bonfires.
Fast Travel:  Once you've discovered a bonfire, you can often warp ("fast travel") to it from other bonfires, allowing you to quickly move around the game world.
Enemy Reset: Resting at a bonfire respawns all non-boss enemies in the area.

          Bonfires are crucially important to survival in the notoriously difficult Dark Souls games. They’re safe havens in a dangerous world, but they also come with strategic considerations due to the enemy respawn mechanic.

          Not bad actually. Even recognized that it is the Remastered version of Dark Souls 1, which is correct.

          AI generated content

          User: What do I do here?

          Gemma 3 27b: Okay, you’re facing the Taurus Demon in Dark Souls! This is a classic early-game boss encounter. Here’s a breakdown of what to do:

          Understanding the Fight

          Big and Slow: The Taurus Demon is very large and, despite looking intimidating, is relatively slow in its attacks.
Weak Point:  His weak point is the glowing, fleshy area on his belly.  You need to hit this to do significant damage.
The Arena: You're in a ruined chamber with pillars. These pillars are important for dodging and can be used to your advantage.
Falling Rocks: Be aware that during the fight, rocks will fall from the ceiling.  They can do significant damage, so keep moving.

          Strategies to Beat Him

          Lure Him to the Stairs: This is the most common and easiest strategy.  Run around the arena to get the Taurus Demon to follow you.  He'll try to smash you with his hammer.  You want to lead him up the stairs on either side of the arena. 
 

Attack the Belly: Once he's on the stairs, lock onto him.  He will have trouble effectively attacking you while on the stairs. This is your chance to repeatedly attack his glowing belly.  Don't get greedy – a few hits, then back off to avoid his counterattacks. 
 

Dodge/Block: When he does swing his hammer, dodge towards the attack (roll into it).  Alternatively, you can block if you have a shield with good stability, but dodging is generally better for this fight. 
 

Watch for the Falling Rocks: Keep moving!  You'll hear a warning sound before they fall. 
 

Second Phase (after roughly 50% health): He will pull out an axe. The strategy is similar, but he'll be even more aggressive. Continue luring him to the stairs and attacking his belly.

          Specific Tips

          Equipment:  Anything you have at this point will work, but a decent shield is helpful for blocking if you're not comfortable dodging.
Humanity:  If you're in human form, you'll get more souls when you beat him, but you'll also be vulnerable to invasion. Hollow form has some resistance to invasions.
Don't Panic: It can be intimidating the first time, but stay calm, observe his patterns, and focus on the strategy.

          In summary, your main goal is to lure the Taurus Demon onto the stairs and attack his belly. Focus on dodging, and be mindful of the falling rocks.

          Good luck! Let me know if you get stuck or have more questions. Do you want me to elaborate on any of these points, or provide a strategy for a specific weapon build?

          Nope, that is the Asylum Demon not the Taurus Demon. The description of the Taurus Demon is also completely wrong.

          I also asked ChatGPT on the same picture:

          AI generated content

          User: What do I do here?

          ChatGPT 5: You are in the Asylum Demon boss fight from Dark Souls (tutorial area).

          Options:

          Do not fight yet. Turn left and run through the small door — visible under the torch on the left wall. This is the intended path. You’ll get a real weapon and shield before returning to kill the demon.

          If you stay: You can damage it slightly, but with your current gear (broken sword) you’ll do almost nothing and likely die.

          After escape: Proceed through the asylum, collect the Estus Flask and Undead Asylum F2 East Key, then return to the boss room by dropping from above for a plunging attack to start the real fight.

          Go left now.

          That’s actually a pretty accurate description on what to do. You could work with that.

          • utopiah@lemmy.worldEnglish
            2·
            8 hours ago

            Thanks again. Well the first sentence started so good, correct game, neat,… but then wrong weapon… so totally pointless.

            Again this can eventually be fixed. It’s “just” a data problem, and that’s exactly what models (and the entire infrastructure of data centers and researchers funded by VC money) excel at. So I think one can safely bet it will get there.

            But… today, can one genuinely imagine playing Dark Souls (or any other game) without… knowing it? Like how does search for the wrong weapon and sometimes the right one help? How is that more convenient that picking a weapon up the searching manually for its name on desktop or mobile knowing with 99% certainty it will be the right one and advice will be genuine and relevant?

            • Domi@lemmy.secnd.meEnglish
              1·
              7 hours ago

              But… today, can one genuinely imagine playing Dark Souls (or any other game) without… knowing it?

              Absolutely not, but I would love to see someone do a challenge run where they play through the entire game just with Copilot. No thinking allowed.

              • utopiah@lemmy.worldEnglish
                2·
                7 hours ago

                lol, performance art. That’d be interesting. I’d watch too.

                Plus 20K people did watch a fish play Pokemon.

                Also now that I think about it, it shouldn’t be too hard to feed a vision model a specific subset, e.g. https://darksouls.wiki.fextralife.com/Weapons of the visuals of all equipment and only then give advice. There is so much hierarchical information in there, e.g. one doesn’t get an Elden Ring weapon in Dark Souls, or does not get an end of the game weapon (except with glitches) after 1h of play time, etc so it’s possible to narrow the search space a lot.

                I imagine a lot can be done with just few curated sources. Now… again (and I apologize for repeating myself so much while possibly sounding pedantic), why? Like what’s the actual point?

              • Hudell@lemmy.dbzer0.comEnglish
                21·
                7 hours ago

                Do you mind trying it with some random skyrim door puzzle or something like that? I imagine it’s closer to what they expect people would use it for, but I don’t imagine it handling any better.

                The whole thing is specially useless if you consider that people are much less likely to try it on games that are old enough for the AI to have had data about it than on brand new games that they know nothing about.

                • Domi@lemmy.secnd.meEnglish
                  2·
                  7 hours ago

                  Stolen from https://www.eurogamer.net/skyrim-golden-claw-door-puzzle-solution-bleak-falls-barrow :

                  Gemma 3 27b:

                  AI generated content

                  User: Can you solve this puzzle for me?

                  Gemma 3 27b: Okay, this is the “Blessing of Akatosh” puzzle from The Elder Scrolls V: Skyrim in the Temple of Kynareth.

                  Here’s the solution:

                  You need to activate the stones in the order representing the stages of a dragon’s life cycle. From left to right, these represent:

                   Egg:  The stone with the coiled serpent (looks like a dragon egg)
 Hatchling: The stone with the dragon head emerging from a spiral.
 Dragon: The stone with the full, flying dragon.

                  So, the order you need to press the stones is: Left, Middle, Right.

                  Activating them in this order will open the pathway to the Blessing of Akatosh. Good luck! Let me know if you get stuck elsewhere.

                  Correct game, wrong puzzle. Did detect that it is a puzzle though.

                  ChatGPT:

                  AI generated content

                  User: Can you solve this puzzle for me?

                  ChatGPT: This image shows the pillar puzzle from Bleak Falls Barrow in The Elder Scrolls V: Skyrim.

                  The correct order of symbols for this puzzle is:

                  Snake – Snake – Fish (Whale).

                  Explanation: If you look around the room before the lever, you’ll see a broken pillar lying on the ground above the gate. It shows the solution pattern in that exact order. Set the three rotating pillars to match:

                  Left pillar → Snake

                  Middle pillar → Snake

                  Right pillar → Fish (Whale)

                  Then pull the lever — the gate will open.

                  According to the Eurogamer article, that is correct.

    • halfapage@lemmy.worldEnglish
      9·
      13 hours ago

      it’s funny how they basically present games as a chore you need to do and need all means you can get to just get through them

      if it’s unpleasant to figure out and play why bother?

      • utopiah@lemmy.worldEnglish
        2·
        12 hours ago

        Spot on, it’s a variation of the “Robots were supposed to clean the house why I make art, not the other way around” meme and arguably Moravec’s paradox.