simulate-mouse-demo
Task
Run the SimulateMouse demo scenario: $ARGUMENTS
What
Automate the SimulateMouse demo scene by clicking buttons, one-shot dragging boxes, split-dragging a box through multiple waypoints to the drop zone, and operating the virtual pad with DragStart/DragMove/DragEnd. Exercises Click, Drag, DragStart, DragMove, and DragEnd across all interactive elements.
When
Use when you need to:
- Run the simulate-mouse demo to verify click, drag, split-drag, and virtual pad functionality
- Test mouse simulation on the demo scene after code changes
- Exercise the demo scene end-to-end (buttons + one-shot drag + split drag + virtual pad)
How
Prerequisites
- Unity must be running with SimulateMouseDemoScene loaded
- PlayMode must be active
If PlayMode is not active, start it with uloop control-play-mode --action Play and wait a moment for the scene to initialize.
Step 1: Discover UI element coordinates
Take an annotated screenshot to get exact coordinates for each interactive element:
uloop screenshot --capture-mode rendering --annotate-elements true
From the AnnotatedElements array in the response, extract SimX and SimY for:
- ClickButton1 — red button
- ClickButton2 — blue button
- DropZone — the drag target area
- RedBox — red draggable box
- GreenBox — green draggable box
- BlueBox — blue draggable box
- LongPressButton — purple long-press button
VirtualPadBackground is also listed in the AnnotatedElements output (Type: "Draggable"). Extract its SimX and SimY directly — these are used as <Pad.SimX> and <Pad.SimY> in the command examples below.
Step 2: Click buttons, one-shot drag, and split drag — chain all in one Bash call
IMPORTANT: Chain all commands with && in a single Bash tool call to eliminate round-trip latency between operations. The simulate-mouse tool uses a single-pointer model, so commands must run sequentially (not in parallel), but chaining avoids AI round-trip delays.
Phase 1 — Button clicks: Alternate ClickButton1 and ClickButton2 four times.
Phase 1.5 — LongPress: LongPress on LongPressButton for 3 seconds. The button gradually changes color during the hold. On first activation it turns orange and shows "Activated!"; on second activation it toggles back to purple and shows "Hold 3s".
Phase 2 — One-shot Drag: Drag RedBox to DropZone top area (Y offset -80 from center) so it doesn't overlap subsequent drops.
Phase 3 — Split drag with DragMove: Use DragStart on GreenBox, then DragMove through 3 waypoints that zigzag across the screen, and finally DragEnd at the DropZone center. Use --drag-speed 400 for DragMove/DragEnd so the movement is slow enough to observe the cursor overlay and trail line.
Waypoint design (relative to the midpoint between GreenBox and DropZone):
- Waypoint 1: far right, slightly above — tests horizontal sweep
- Waypoint 2: far left, below — tests direction reversal
- Waypoint 3: above DropZone — tests vertical approach
- DragEnd: at DropZone center (Y offset 0)
Phase 4 — One-shot Drag: Drag BlueBox to DropZone bottom area (Y offset +80 from center) so it doesn't overlap previous drops.
Phase 5 — Virtual Pad: Use DragStart on VirtualPadBackground center, then DragMove through 8 random directions within padRadius (80px from center) to exercise the joystick, and DragEnd back at center. Use --drag-speed 300 for visible movement. The direction arrow should rotate to follow each DragMove direction.
Waypoint design (offsets from VirtualPadBackground center):
- Move 1: upper-right (+60, -60 in sim coords)
- Move 2: lower-left (-70, +50)
- Move 3: straight up (0, -75)
- Move 4: straight right (+80, 0)
- Move 5: lower-right (+50, +60)
- Move 6: straight left (-80, 0)
- Move 7: upper-left (-55, -65)
- Move 8: straight down (0, +75)
- DragEnd: back to center (0, 0)
uloop simulate-mouse-ui --action Click --x <ClickButton1.SimX> --y <ClickButton1.SimY> && sleep 0.3 && \
uloop simulate-mouse-ui --action Click --x <ClickButton2.SimX> --y <ClickButton2.SimY> && sleep 0.3 && \
uloop simulate-mouse-ui --action Click --x <ClickButton1.SimX> --y <ClickButton1.SimY> && sleep 0.3 && \
uloop simulate-mouse-ui --action Click --x <ClickButton2.SimX> --y <ClickButton2.SimY> && sleep 0.3 && \
uloop simulate-mouse-ui --action LongPress --x <LongPressButton.SimX> --y <LongPressButton.SimY> --duration 3.0 && sleep 0.3 && \
uloop simulate-mouse-ui --action Drag --from-x <RedBox.SimX> --from-y <RedBox.SimY> --x <RedBox.SimX> --y <DropZone.SimY - 80> --drag-speed 700 && sleep 0.3 && \
uloop simulate-mouse-ui --action DragStart --x <GreenBox.SimX> --y <GreenBox.SimY> && sleep 0.3 && \
uloop simulate-mouse-ui --action DragMove --x <DropZone.SimX + 150> --y <GreenBox.SimY - 50> --drag-speed 400 && sleep 0.3 && \
uloop simulate-mouse-ui --action DragMove --x <DropZone.SimX - 150> --y <DropZone.SimY + 50> --drag-speed 400 && sleep 0.3 && \
uloop simulate-mouse-ui --action DragMove --x <DropZone.SimX> --y <DropZone.SimY - 80> --drag-speed 400 && sleep 0.3 && \
uloop simulate-mouse-ui --action DragEnd --x <DropZone.SimX> --y <DropZone.SimY> --drag-speed 400 && sleep 0.3 && \
uloop simulate-mouse-ui --action Drag --from-x <BlueBox.SimX> --from-y <BlueBox.SimY> --x <BlueBox.SimX> --y <DropZone.SimY + 80> --drag-speed 700 && sleep 0.3 && \
uloop simulate-mouse-ui --action DragStart --x <Pad.SimX> --y <Pad.SimY> && sleep 0.3 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX + 60> --y <Pad.SimY - 60> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX - 70> --y <Pad.SimY + 50> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX> --y <Pad.SimY - 75> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX + 80> --y <Pad.SimY> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX + 50> --y <Pad.SimY + 60> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX - 80> --y <Pad.SimY> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX - 55> --y <Pad.SimY - 65> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragMove --x <Pad.SimX> --y <Pad.SimY + 75> --drag-speed 300 && sleep 0.4 && \
uloop simulate-mouse-ui --action DragEnd --x <Pad.SimX> --y <Pad.SimY> --drag-speed 300
Step 3: Verify results
Take a final screenshot to confirm all boxes reached the drop zone:
uloop screenshot --capture-mode rendering
More from hatayama/unity-cli-loop
uloop-control-play-mode
Control Unity Editor play mode (play/stop/pause). Use when you need to: (1) Start play mode to test game behavior, (2) Stop play mode to return to edit mode, (3) Pause play mode for frame-by-frame inspection.
3uloop-run-tests
Execute Unity Test Runner and get detailed results. Use when you need to: (1) Run EditMode or PlayMode unit tests, (2) Verify code changes pass all tests, (3) Diagnose test failures with error messages and stack traces. Auto-saves NUnit XML results on failure.
2uloop-compile
Compile Unity project and report errors/warnings. Use when you need to: (1) Verify code compiles after C# file edits, (2) Check for compile errors before testing, (3) Force full recompilation with Domain Reload. Returns error and warning counts.
2uloop-screenshot
Capture screenshots of Unity Editor windows as PNG files. Use when you need to: (1) Screenshot Game View, Scene View, Console, Inspector, or other windows, (2) Capture current visual state for debugging or documentation, (3) Save editor window appearance as image files.
2uloop-launch
Launch Unity project with matching Editor version via uloop CLI. Use when you need to: (1) Open a Unity project with the correct Editor version, (2) Restart Unity to apply changes, (3) Switch build target when launching.
2uloop-get-logs
Retrieve logs from Unity Console with filtering and search. Use when you need to: (1) Check for errors or warnings after compilation or play mode, (2) Debug issues by searching log messages, (3) Investigate failures with stack traces. Supports filtering by log type, text search, and regex.
2