Remote control your Android phone through adb

I wrecked my phone the other day:

Broken PhoneDoesn’t look so bad, but in fact the touch screen is completely dead. So not only do I now have to decide whether to get a new phone or try to replace the screen myself, I also had to find a way to regain control over the phone so I could backup my data.

A bluetooth mouse would be one way to control the phone without the touch screen, but I don’t have one – and even if I did, I didn’t have bluetooth turned on (though that could apparently be accomplished through the console).

A USB keyboard would also be of help if the Nexus 4 had USB Host support. Sadly, unlike the Nexus 5 (and the Nexus 1, if I recall correctly), doesn’t.

What else then? A series of internet searches yielded the following results:

  1. There’s a ton of remote control apps for Android (as was to be expected). Most require root though. Those that don’t generally can’t control the touchscreen/mouse. And even they generally require some setup steps before they can be used – back to the start.
  2. adb (the Android Debug Bridge, part of the SDK tools) provides shell access to the phone via USB. And the shell has commands to simulate key presses and touch events. Touch events require the exact screen coordinates as numbers, so as a standalone solution that’s still rather tedious.
  3. There are ways to get screenshots from an Android phone through adb – obviously; after all, even the official Eclipse plug-ins provide a screenshot feature.
  4. A combination of points 2 and 3 would be a way to actually simulate touch events in a useful manner. Astonishingly though, nobody seems to have written such a tool.

So what does a programmer to when the obvious tool doesn’t yet exist? Write it himself, of course.

ADB Control

I spent yesterday evening hacking together a crude Java tool to serve as a primitive remote control through an Android phone. I present it here in case someone else might face the same problem I did. It requires no extra software installed on the phone, just a USB connection and the Java tool (Java 7 required) and SDK tools installed on the computer. It uses adb access to the phone – meaning that you have to have the SDK tools installed and you have to have already confirmed the USB debug connection on your phone earlier. If you didn’t, connecting the phone will open a confirmation dialogue on the phone – and without a touchscreen, you obviously won’t be able to confirm it. But if you’re a programmer like me and have already access the phone via adb before, all you have to do is fire up the tool on the computer. I won’t get into how to setup and configure adb here – please check the official docs if necessary.

The tool does the following:

  • Repeatedly grab a screenshot from the phone and display it in a Window
  • Capture keyboard events and forward them to the phone
  • Capture mouse clicks and drags and forward them to the phone

It’s slow as hell (everything is being done through shell calls, which makes the screenshot updates particularly slow), but it works.

Before you first start it, make sure you have adb installed and modify the config file provided with the download. It contains four settings:

  • adbCommand – the full path to the adb tool as a shell command. Linux users note that Java doesn’t seem like the ~ character in file paths to access the home directory.
  • screenshotDelay – the delay (in milliseconds) between displaying a screenshot and grabbing the next one. Note that the actual grabbing process will probably take considerably longer than this delay, so the update rate will be slower.
  • localImageFilePath – the tool has to download the image file from the phone. This path tells it where to store the file on the local system before displaying it.
  • phoneImageFilePath – the location on the phone where the screenshot file is stored before downloading it.

Once the configuration is done, simply run the tool like this:

java -jar adbcontrol.jar

If you put the config file in a different directory, you can add the path to it as an option after the jar file name.

Here’s what it looks like:

ADB ControlIn the window, you can single click to send a tap event or click-drag to send a swipe event to the phone. Pressing a number or letter key (or anything else that produces an ASCII character between 33 and 127) will send that key press to the phone. Additionally, the following keys are forwarded:

  • Enter
  • Escape – as the Back key on Android
  • Home
  • Backspace
  • Cursor keys

Download

The download includes the full source code and is provided under the Eclipse Public Licence.




In case you find this tool useful, please consider donating – after all, I only wrote it because I wrecked my phone.

This entry was posted in Android, Uncategorized and tagged , , . Bookmark the permalink.

78 Responses to Remote control your Android phone through adb

  1. coleen says:

    how do you modify the config file? Dos prompt?

    • It’s just a text file. You can use any text editor.

      • Ismael says:

        These would be much saner defaults:

        adbCommand = adb
        screenshotDelay = 100
        localImageFilePath = /tmp/adbcontrol_screenshot.png
        phoneImageFilePath = /mnt/sdcard/adbcontrol_screenshot.png

        Otherwise, freakishly awesome application, thanks!

  2. J. Reis says:

    This is excellent. I cannot thank you enough for having the forethought to post this for those unfortunates who might be searching for a solution. My screen worked, but without touch functionality. I knew enough to use adb to input touch events, but 15 minutes of guessing coordinates and I had just managed to mute the blaring alarm and copy out my open browser tabs. The thought of muddling through my apps to jot down settings, etc was not a pleasant one.

    Your tool made the rest of my night a breeze. Less than an hour of navigating around and with mouse and keyboard and I’ve got everything copied I need and the peace of mind of having been able to check every nook and cranny for info I might want. Off to get a new phone tomorrow, I owe my good sleep tonight to you sir!

    Sincerest thanks

  3. Jeremy says:

    You’re an F’in genious man. I knew adb could do “input tap x y”, but you took it a step further. My screen’s busted, but the display still comes on. I just can’t use touch. I was surprised to see tap and drag worked well also. Major kudos to you, man. There’s 15 euros for your efforts.

  4. humulos says:

    I can’t wait to try this! My nexus 5 got run over, shattering the screen, but the internals are working fine. I needed a way to pair it with a Bluetooth keyboard so I could then plug into my TV with slimport, and this looks like it will work! Homemade set top box here I come!

  5. Azeem says:

    I don’t understand the config file still. I’m using windows so my adb path is c:\android\adb.exe what am i supposed to put as the path in the config file?

    • I’d use double backslashes, as advised by Sai. Windows requires backslahes (Java generally does not, but I execute adb through a native shell command, so Windows has to handle the path). But the .properties file format uses the backslash as an escape character, so you have to double it to get an actual backslash in the path.

  6. Sai says:

    thanks a lot, Marian for this code.
    @Azem, try using “C:\\android\\adb”, for me it worked on windows 8 with java 1.7.

  7. Vlad says:

    Hi,

    Good app, but swiping the status bar is not working well under Android 4.2 – it looks closing from the opened state all the time, but cannot stay opened. Log window shoes that “Swipe from …to…” was detected well.

    And remark – would be nice if any progress is visible during waiting for the next screenshot.

    • That’s odd, because I really send just the swipe action seen in the log to the phone. It worked on my Nexus. Is this with stock Android?

      Please feel free to modify the code and add some kind of feedback during screenshots. I don’t intend to do any more work on this myself; I just hacked it together quickly to get into my own broken phone. To be honest, I was a bit disappointed that the delay between screenshots was *so* long – but at the same time, I was happy that it worked at all.

  8. Сергей says:

    May i unlock the locked screen by this app?
    Thanks.

    • If you use a PIN or password based screen lock, it shouldn’t be a problem (I’ve used it with PIN lock myself).

      The swipe figure based screen lock will probably pose problems though, because the tool doesn’t handle polygon swipes. A swipe gesture is just sent to the phone as a single linear swipe from the start to the end position, so anything more complex than a straight line won’t work.

  9. seaders says:

    Seriously can’t thank you enough for this. Needed to have a test device on Lollipop and the only one available (my old Nexus 4) had a big brokened screen. As soon as I got past a few adb issues this worked a charm, seriously awesome altogether!

  10. Pingback: Displayschaden: Android-Smartphone über ADB steuern - Wlanowskis Webblog

  11. Grtschnk says:

    Thanks! 😀
    Great idea, just got it to work. Any chance of supporting landscape mode? http://stackoverflow.com/questions/10040624/check-if-device-is-landscape-via-adb

    Sorry for already suggesting features,,

    • I don’t really plan on doing any more work on this tool; after all, I only wrote it out of necessity in the first place. But feel free to add it yourself if you feel like it.

      Considering that one of the comments says the command needs some time to execute, it probably won’t really help to repeatedly execute it (taking screenshots via adb is slow enough as it is). You could check the orientation only once during startup and then stick to it – or add some button or keyboard shortcut to manually trigger an orientation update. In that case it might be simpler (if not quite as elegant) to simply let the user change the window orientation manually though.

  12. Chris says:

    I have no idea what im doing everytime i run the tool from cmd promp it says unable to access pls help and have like some step by step instructions pls.

  13. Rafa says:

    Wonderfull!!
    I only have one problem with my NExus 5, Space doesn’t works!

  14. Anton says:

    Great idea

    What license of code?
    What you think about post it to github?

    • David says:

      I’m also curious about this.. would you have a problem if we uploaded it to github and extended the features?

      • (Sorry to Anton for apparently missing his original comment).

        As mentioned in the post, the licence is the Eclipse Public Licence (https://www.eclipse.org/legal/epl-v10.html), which allows you to freely modify and distribute the program as long as you stick to the licence and publish the source code. I certainly wouldn’t mind it getting a new live on github (if I did, I would have picked a different licence), and I’d be interested in seeing what people can make of it. My version was hacked together in one evening, so I’m sure there are better ways to do things, and I’d expect a serious overhaul would soon have replaced most of my original code. Speeding up the screenshot interval would be helpful. I imagine you could even automate the tool to upload an APK to the phone and have it act as a server for some operations (e.g. compressed screenshots or something).

        So: Go for it, and good luck! :)

  15. Trevor (uplusion23) says:

    Thanks a ton man! I have a screen that stopped display and all! Luckily I have no PIN lock or anything. With this I installed Teamviewer QuickSupport on my Galaxy S3, and could control it that way.

  16. Pingback: My Inflation RPG Plays Harvester | A Dad in Vancouver

  17. Thanks a lot! This saved my life, since I couldn’t get any VNC server running. I have been searching hours for a solution, and this was perfect, thanks!!!

  18. Sam says:

    Hmmm… I *thought* my touch screen was broken, but my phone is ignoring inputs from this app too. Why is my phone ignoring me ?! Or maybe it just doesn’t work with the lock screen?

  19. Buda Florin says:

    Saved my day! Thanks !

  20. bDroid says:

    This is great… took a bit of troubleshooting, but thanks to all the comments to help get this working. I’m curious if it’s an issue to forward ASCII chars 32 and up? The space (space-bar) doesn’t get forwarded and is somewhat annoying. Cheers!

  21. João Victor says:

    help me use windows , ( pt-br )
    only opens a window adbcontrol , adb shell commands funciondo , java 8

  22. Hey, I tried to use it, my condition is worse than yours … my screen is off the device itself

    When i try to run the utility it yells out saying “Can’t read input file”

    any help ?

  23. Adrien Turcey says:

    You’re awesome, my One Plus One touchscreen is broken and thanks to you, i can control it with my mouse.

    Thanks a lot!!!!!

    My config file for others (For windows) :

    adbCommand = C:/android-sdk/platform-tools/adb
    screenshotDelay = 100
    localImageFilePath = C:/Users/Adrien/Desktop/test/img.png
    phoneImageFilePath = /mnt/sdcard/adbcontrol_screenshot.png

  24. James says:

    Can I somehow rotate the image when my phone is in Landscape mode?

  25. Guille says:

    I dont have an sdcar, can you tell me the internal mount point of the mobilephone?
    Thanks.

  26. Silas says:

    Very awesome, I have a phone with a busted digitizer, so this is exactly what I was looking for to help bring it back to life. Thanks again.

  27. luchoz says:

    Great work!!!! thank you very much!!

  28. Anon says:

    Hey All,

    This is great! Thank you for your hard work :)

    Please use the below settings for this application

    adbCommand = C:\\adb\\adb
    screenshotDelay = 100
    localImageFilePath = c:\\adb\\ping.png
    phoneImageFilePath = /sdcard/Pictures/Screenshots/screeny.png

    **Quick Notes:
    phoneImageFilePath doesnt have to match your phone’s default location. Just a place of your choosing. Same applies to the localImageFilePath

    /sdcard/ is the location for the phone’s INTERNAL storage, so dont worry,it wont start looking for sdcards.

    ADB must be installed. I recommend googling for fast setup (as i am unsure about hotlinking on marian’s blog):
    [TOOL] [WINDOWS] ADB, Fastboot and Drivers – 15 seconds ADB Installer v1.4.2

    Phone should have USB debugging enabled and in PTP mode upon connection. (not sure if it applies to all phones). Please google how to get into developer options to enable this option.

    Cheers :)

  29. Marty says:

    Hi,

    Just trying this out now on my ubuntu set-up. I’m getting the following error, any ideas?!

    javax.imageio.IIOException: Can’t read input file!
    at javax.imageio.ImageIO.read(ImageIO.java:1301)
    at name.schedenig.adbcontrol.AdbControlPanel.loadImage(AdbControlPanel.java:255)
    at name.schedenig.adbcontrol.AdbControlPanel.makeScreenshot(AdbControlPanel.java:248)
    at name.schedenig.adbcontrol.AdbControlPanel.access$0(AdbControlPanel.java:245)
    at name.schedenig.adbcontrol.AdbControlPanel$2.run(AdbControlPanel.java:89)

    • Alexander says:

      This is due to a wrong path to either the phone file or to the local file in the config file.
      Check that both paths exist, that non-alphanumeric characters are properly escaped (or better yet not used at all), and that appropriate permissions are granted (that is, the user you run adbcontrol.jar with can access the local file path).

  30. Alexander says:

    Thank you so much! You saved me today! This is the only app that actually does the job!

    Three improvements could make it even greater:
    1. Support for international character input from PC keyboard (now I can’t type Russian and have to use Android’s on-screen keyboard for that)
    2. Automatic resizing of the java window to the size (25%, 50%, 100%, 200%) of the device screen. Currently it shows some default-sized window that doesn’t match the device size and precisely scaling it is a pain.
    3. A GUI for editing the settings. So that one could just push “browse” and locate ADB in the filesystem. It may be counter-intuitive for most people that Windows paths need escaping (e.g. instead of “C:\Users\myname\AppData…” one has to write “C:\\Users\\myname\\AppData…”). GUI would save people from stumbling at that.

    Anyway, your app is great! Thank you again!

  31. Bob says:

    Wow, this app was so helpful. I cracked my screen, I tried hooking a mouse into the usb but didn’t work. I used this to send my contacts to my new phone. Would have been nice to have a menu key option (I see you commented it out in the code), but adb works just fine for this anyways… Thanks!!

  32. A says:

    If you got a pattern lockscreen this won’t work unfortunately. However, you could connect to ADB wirelessly. See: http://developer.android.com/tools/help/adb.html#wireless

    Then you buy a usb-otg adapter for your phone and use it to connect a mouse.

    This way you can take repeated screenshots/videos and soon you will have managed to mark the right pattern. http://www.learn2crack.com/2014/08/capture-screenshot-record-screen-using-adb.html

    If your screen isn’t broken, then you only need a mouse and a otg-adapter ofcourse.

  33. Pingback: Device suddenly unauthorized for adb | Asking

  34. This is great.
    Saved me a lot of time.
    Using it to send key events to Cardboard VR (very nifty).

  35. phyatt says:

    Make sure you are running adbcontrol.jar from a path without spaces. This is my preferred config file for Windows:

    adbCommand = C:/Android/SDK/platform-tools/adb.exe
    screenshotDelay = 100
    localImageFilePath = adb.png
    phoneImageFilePath = /mnt/sdcard/adbcontrol_screenshot.png

    With adbcontrol.jar and config.properties under C:/Android

  36. phyatt says:

    Make sure you install to a path without spaces. Here is the way I like to use it on Windows:

    adbCommand = C:/Android/SDK/platform-tools/adb.exe
    screenshotDelay = 100
    localImageFilePath = adb.png
    phoneImageFilePath = /mnt/sdcard/adbcontrol_screenshot.png

    And I install to C:/Android/

  37. Arpan Das says:

    Thanks a lot.

    This config worked for me:
    =====================
    adbCommand = adb
    screenshotDelay = 100
    localImageFilePath = temp/snap.png
    phoneImageFilePath = /mnt/sdcard/adbcontrol_screenshot.png

  38. Geoffrey says:

    This is great, how can we get space bar to be forwarded? from the ADB shell it’s %s. Any way to have the program send a %s when you press space?

  39. Pingback: Device suddenly unauthorized for adb | Question and Answer

  40. Harish says:

    This is exactly what I wanted today!

  41. Pingback: Second life of a Smartphone | roboquark

  42. kert says:

    Thanks for writing this – saved my bacon too, and here is the small donation.
    Everything on the phone is backed up anyway, but cracked/broken touch screen and a number of annoying little things need rescuing from multiple apps, and this is just enough to get the job done and easy to run. Also used the unlock.sh script to get past the pattern screen lock.
    There are other solutions like VMLite VNC server, but thats $10 and not much help when its not already installed.
    Seems like a such an obvious tool – doesnt Android studio or some Eclipse thing come with something like this built in already ?

  43. Ntropia says:

    “Kudos for writing this in the first place, but also for sharing it with the community!”
    — another satisfied customer that found exactly what he was looking for

  44. ravi kumar says:

    java jar adbcontrol.jar
    Error: Could not find or load main class jar

  45. Pete says:

    How have people mounted their phone? I’m not able to find it in /mnt/

    $adb devices
    List of devices attached
    39069840 offline

    $lsusb
    Bus 002 Device 012: ID 04e8:6860 Samsung Electronics Co., Ltd GT-I9100 Phone [Galaxy S II], GT-I9300 Phone [Galaxy S III], GT-P7500 [Galaxy Tab 10.1]

  46. Pete says:

    My phone is using mtp://

    Is there a way to prevent using mtp, I assume this is why it won’t mount properly. I’m getting this error:

    javax.imageio.IIOException: Can’t read input file!
    at javax.imageio.ImageIO.read(ImageIO.java:1301)
    at name.schedenig.adbcontrol.AdbControlPanel.loadImage(AdbControlPanel.java:255)
    at name.schedenig.adbcontrol.AdbControlPanel.makeScreenshot(AdbControlPanel.java:248)
    at name.schedenig.adbcontrol.AdbControlPanel.access$0(AdbControlPanel.java:245)
    at name.schedenig.adbcontrol.AdbControlPanel$2.run(AdbControlPanel.java:89)

  47. Pete says:

    Further more, I can access the phone like this, and create a dir ‘temp’:
    /run/user/1000/gvfs/mtp:host=%5Busb%3A002%2C054%5D/Phone/temp

    But I’m not allowed to create new files in there.

    $sudo touch test.txt
    touch: cannot touch ‘test.txt’: Permission denied

    Is this the error that is coming back?

  48. PutterPlace says:

    For those of you wondering how to get spaces to work, just add an extra case to the switch in keyPressed( … ):

    case KeyEvent.VK_SPACE:
    adbHelper.sendKey(AndroidKey.SPACE);
    return;

    Simple as that. :-)

  49. Anthony says:

    You saved my life! Thank you!

  50. karolba says:

    Thank you, that’s really useful!

  51. dcdh says:

    It works like a charm.

    You saved me !

    Thank you ^^

    Damien

  52. Hartwig says:

    Man, you rock! This is AMAZING!!! Thanks a lot!

  53. Emgi says:

    Works very good!

  54. David says:

    Hey, great tool. It’s amazing what you can do with a few lines of code if you have a good idea.
    Unfortunately my wrecked Nexus 4 is not in adb mode, so it does not respond to the adb commands.

  55. Psolyca says:

    Thanks a lot !

    You made my day, I was doing the same command by command.
    I have installed AirDroid for more control.

  56. Pingback: android-Handy runtergefallen, touch-Screen kaputt? wenn adb aktiviert und die Verbindung mit dem Rechner…

  57. Jordi Babot says:

    So if I’m not wrong, if I have my screen broken (no haptic at all) and don’t have USB debugging ON, then this will not work… cause I can’t access the option :(

    So sad! :(
    fortunatelly I could access my photos at least.
    Good job though!

  58. david says:

    I’ve spent days trying to modify this file. but ahhhhhhhhhhhhhhhhhhhhh…

    I can’t even compile this code T.T

    can someone plzz help plz…

    i just want to change screen size

  59. david says:

    nvm i solved how to compile and recompile

    now i need to disable portrait mode and replace it with landscape mode!!

    any help is appreciated

    email: 26212496k@gmail.com

  60. Andrei-Marius says:

    One word: “Priceless…”
    I hope you find a tutorial like this that saves you a whole lot of time doing research and debugging.
    Cheers mate :)

  61. S.S.Haider says:

    I am having trouble that I am getting click and scroll messages on console but nothing happening on phone ,also I am unable to get images from phone and getting imageio.IIOException: Can’t read input file, however i simply put an image at the specified directory than it is showing that image and at least program is not crashing but overall program is not functioning properly

  62. Man, good work. Second wreckage of my phone this year, I’m tapping like crazy via adb shell input tap, to turn on wifi and similar stuff you had to do, but this makes it peace of cake. Thanks.

    Michal, Slovakia

  63. Pierre-Luc says:

    You, Sir, are a Genius.

  64. Lovish Jain says:

    I don’t have any words to appretiate your work. Its damn awesome. Exactly what I wanted from last 8 months. I too tried to study and make a software like this but mine didn’t work atall.
    Thank you so much for providing it.

  65. Chauntelle says:

    Has anyone donated to the creator of this program? How & where to do so if desired?

  66. Kayanski says:

    Man you just saved my life, giving me hope again.
    But for a better experience try Vysor, a chrome extension here : https://chrome.google.com/webstore/detail/vysor/gidgenkbbabolejbgbpnhbimgjbffefm
    It works all the same but it is slightly faster (for me at least)

  67. Heinz says:

    Dear Marian, I would like to express my huge appreciation to you for your excellent work!

Leave a Reply

Your email address will not be published. Required fields are marked *