引文：

因為我棄用原來ATX框架中的uiautomator的東西，所以現在要把 UiSelector().text("XXX")這部分的功能給重新實現下。

所以這篇文章介紹的是抓取到頁面中的文字還有文字的位置的方法及其分析。

現有的方法

1，UiSelector.text /自動化測試框架

大多數測試框架使用的方法。好像需要在手機上安裝一個測試的app，沒動手實踐

2，Layout Inspector

android studio->Tools->android->Layout Inspector

顯示很友好，沒找到開源部分的代碼

3，uiautomator dump

adb shell uiautomator dump

dump出來的東西會自動保存在/sdcard/window_dump.xml中，內容大概是這樣

...

resource-id="com.android.contacts:id/contacts_unavailable_view"

class="android.widget.FrameLayout"

package="com.android.contacts"

content-desc="" checkable="false" checked="false"

clickable="false" enabled="true" focusable="false"

focused="false" scrollable="false" long-clickable="false"

password="false" selected="false"

bounds="[0,408][1080,1776]">

resource-id="com.android.contacts:id/contacts_unavailable_container" class="android.widget.FrameLayout"

package="com.android.contacts"

content-desc="" checkable="false" checked="false"

clickable="false" enabled="true" focusable="false"

focused="false" scrollable="false" long-clickable="false"

password="false" selected="false"

bounds="[0,1776]">

resource-id="com.android.contacts:id/floating_action_button"

class="android.widget.ImageButton"

package="com.android.contacts"

content-desc="添加新聯系人" checkable="false"

checked="false" clickable="true"

enabled="true" focusable="true" focused="false"

scrollable="false" long-clickable="false"

password="false" selected="false"

bounds="[864,1560][1032,1728]"/>

...

...

數據類型都是類似的，都是node節點，bounds就是各個view的邊界了，舉個例子

添加新聯系人

bounds= 864,1560 1032,1728

所以重心就是 (948，1644)

adb shell input tap 948 1644

就點擊到文字了。

ok，下面也是談實現細節。dump調用鏈是這樣的

DumpCommand:run

-> automationWrapper.getUiAutomation().getRootInActiveWindow()

-> AccessibilityInteractionClient:findAccessibilityNodeInfoByAccessibilityId

-> binder

-> ViewRootImpl.findAccessibilityNodeInfoByAccessibilityId

-> AccessibilityNodeInfoDumper.dumpWindowToFile

序列化的節點node的數據存在這里

AccessibilityNodeInfo.java

可以看到，它是一顆樹的節點

public class AccessibilityNodeInfo implements Parcelable {

...

private static final int MAX_POOL_SIZE = 50;

private static final SynchronizedPool sPool =

new SynchronizedPool<>(MAX_POOL_SIZE);

...

}

節點是這樣建的

view.java

public AccessibilityNodeInfo createAccessibilityNodeInfoInternal() {

AccessibilityNodeProvider provider = getAccessibilityNodeProvider();

if (provider != null) {

return provider.createAccessibilityNodeInfo(AccessibilityNodeProvider.HOST_VIEW_ID);

} else {

AccessibilityNodeInfo info = AccessibilityNodeInfo.obtain(this);

onInitializeAccessibilityNodeInfo(info);

return info;

}

}

public void onInitializeAccessibilityNodeInfoInternal(AccessibilityNodeInfo info) {

...

getDrawingRect(bounds);

info.setBoundsInParent(bounds);

}

在TextView

Override

public void onInitializeAccessibilityNodeInfoInternal(AccessibilityNodeInfo info) {

super.onInitializeAccessibilityNodeInfoInternal(info);

final boolean isPassword = hasPasswordTransformationMethod();

info.setPassword(isPassword);

if (!isPassword || shouldSpeakPasswordsForAccessibility()) {

info.setText(getTextForAccessibility());

}

...

}

所以AccessibilityNodeInfo更像是在View中存了一個副本，這個副本可以用于輔助操作。

// TODO 更細節的分析

4,hierarchyviewer1

這個是在源碼中找到的工具，發現它抓取的信息出乎意料的很全，比3中的信息還要多許多，是這樣用的

.out/host/linux-x86/bin/hierarchyviewer1

是一個gui工具，打開之后，選中某個元素，可以看到Property中有absolute_x、absolute_y、getHeight、getWidth、mText。更加這些信息我們可以判斷找的這個元素是不是在屏幕內，在的話，具體是哪個位置。

/sdk/hierachyviewer/ com.android.hierarchyviewer.scene.ViewHierarchyLoader

package com.android.hierarchyviewer.scene;

public class ViewHierarchyLoader {

public static ViewHierarchyScene loadScene(IDevice device,Window window) {

...

System.out.println("==> Starting client");

socket = new Socket();

socket.connect(new InetSocketAddress("127.0.0.1",

DeviceBridge.getDeviceLocalPort(device)));

out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));

in = new BufferedReader(new InputStreamReader(socket.getInputStream(),"utf-8"));

System.out.println("==> DUMP");

out.write("DUMP " + window.encode());

out.newLine();

out.flush();

Stack stack = new Stack();

boolean setRoot = true;

ViewNode lastNode = null;

int lastWhitespaceCount = Integer.MAX_VALUE;

while ((line = in.readLine()) != null) {

// debug by yeshen：

// System.out.println(line);

if ("DONE.".equalsIgnoreCase(line)) {

break;

}

...

}

...

}

}

看樣子是自己建了一個socket取連本地的socket server，在代碼里面打log，發現line的數據是類似這樣的：

android.widget.Button@eb7e4a6

text:mCurTextColor=9,-13224394text:mGravity=2,17

text:mText=6,創建新聯系人

getEllipsize()=4,null

text:getScaledTextSize()=4,14.0

text:getSelectionEnd()=2,-1

text:getSelectionStart()=2,-1

text:getTextSize()=4,42.0

text:getTypefaceStyle()=6,NORMAL

layout:mBottom=3,144

theme:com.android.contacts:style/PeopleTheme()=6,forced

theme:android:style/Theme.DeviceDefault()=6,forced

fg_=4,null

mID=24,id/create_contact_button

drawing:mLayerType=4,NONE

layout:mLeft=1,0

measurement:mMeasuredHeight=3,144

measurement:mMeasuredWidth=3,324

measurement:mMinHeight=3,144

measurement:mMinWidth=3,264

padding:mPaddingBottom=2,30

padding:mPaddingLeft=2,36

padding:mPaddingRight=2,36

padding:mPaddingTop=2,30

mPrivateFlags_DRAWN=4,0x20

mPrivateFlags=9,0x1028830

layout:mRight=3,324

scrolling:mScrollX=1,0

scrolling:mScrollY=1,0

mSystemUiVisibility_SYSTEM_UI_FLAG_VISIBLE=3,0x0

mSystemUiVisibility=3,0x0

layout:mTop=1,0

padding:mUserPaddingBottom=2,30

padding:mUserPaddingEnd=11,-2147483648

padding:mUserPaddingLeft=2,36

padding:mUserPaddingRight=2,36

padding:mUserPaddingStart=11,-2147483648

mViewFlags=10,0x18004001

drawing:getAlpha()=3,1.0

layout:getBaseline()=2,88

accessibility:getContentDescription()=4,null

drawing:getElevation()=3,6.0

getFilterTouchesWhenObscured()=5,false

getFitsSystemWindows()=5,false

layout:getHeight()=3,144

accessibility:getImportantForAccessibility()=3,yes

accessibility:getLabelFor()=2,-1

layout:getLayoutDirection()=22,RESOLVED_DIRECTION_LTR

layout:layout_gravity=4,NONE

layout:layout_weight=3,0.0

layout:layout_bottomMargin=2,45

layout:layout_endMargin=11,-2147483648

layout:layout_leftMargin=1,0

layout:layout_mMarginFlags_LEFT_MARGIN_UNDEFINED_MASK=3,0x4

layout:layout_mMarginFlags_RIGHT_MARGIN_UNDEFINED_MASK=3,0x8

layout:layout_mMarginFlags=4,0x0C

layout:layout_rightMargin=1,0

layout:layout_startMargin=11,-2147483648

layout:layout_topMargin=1,0

layout:layout_height=12,WRAP_CONTENT

layout:layout_width=12,MATCH_PARENT

layout:getLocationOnScreen_x()=3,378

layout:getLocationOnScreen_y()=3,757

measurement:getMeasuredHeightAndState()=3,144

measurement:getMeasuredWidthAndState()=3,324

drawing:getPivotX()=5,162.0

drawing:getPivotY()=4,72.0

layout:getRawLayoutDirection()=7,INHERIT

text:getRawTextAlignment()=7,GRAVITY

text:getRawTextDirection()=7,INHERIT

drawing:getRotation()=3,0.0

drawing:getRotationX()=3,0.0

drawing:getRotationY()=3,0.0

drawing:getScaleX()=3,1.0

drawing:getScaleY()=3,1.0

getScrollBarStyle()=14,INSIDE_OVERLAY

drawing:getSolidColor()=1,0

getTag()=4,null

text:getTextAlignment()=7,GRAVITY

text:getTextDirection()=12,FIRST_STRONG

drawing:getTransitionAlpha()=3,1.0

getTransitionName()=4,null

drawing:getTranslationX()=3,0.0

drawing:getTranslationY()=3,0.0

drawing:getTranslationZ()=3,0.0

getVisibility()=7,VISIBLE

layout:getWidth()=3,324

drawing:getX()=3,0.0

drawing:getY()=3,0.0

drawing:getZ()=3,6.0

focus:hasFocus()=5,false

drawing:hasOverlappingRendering()=4,true

drawing:hasShadow()=4,true

layout:hasTransientState()=5,false

isActivated()=5,false

isClickable()=4,true

drawing:isDrawingCacheEnabled()=5,false

isEnabled()=4,true

focus:isFocusable()=4,true

isFocusableInTouchMode()=5,false

focus:isFocused()=5,false

isHapticFeedbackEnabled()=4,true

drawing:isHardwareAccelerated()=4,true

isHovered()=5,false

isInTouchMode()=4,true

layout:isLayoutRtl()=5,false

drawing:isOpaque()=5,false

isPressed()=5,false

isSelected()=5,false

isSoundEffectsEnabled()=4,true

drawing:willNotCacheDrawing()=5,false

drawing:willNotDraw()=5,false

不過這里缺了 absolute_x absolute_y，找了一下代碼

com.android.hierarchyviewer.ui.model.PropertiesTableModel

private void loadPrivateProperties(ViewNode node) {

int x = node.left;

int y = node.top;

ViewNode p = node.parent;

while (p != null) {

x += p.left - p.scrollX;

y += p.top - p.scrollY;

p = p.parent;

}

ViewNode.Property property = new ViewNode.Property();

property.name = "absolute_x";

property.value = String.valueOf(x);

privateProperties.add(property);

property = new ViewNode.Property();

property.name = "absolute_y";

property.value = String.valueOf(y);

privateProperties.add(property);

}

所以我們已經知道了，上面抓到的數據是一個ViewNode，然后遞歸解析，算出絕對位置。

所以找到文字的位置就要，遞歸算到自己在根視圖的絕對位置，然后再修正下layout:getWidth(),layout:getHeight(),找到view的重點，最后再執行點擊。

ok,下面談實現細節

hierarchyviewer其實是在android上開了一個ViewServer，然后把ViewServer的端口轉發到本地，然后在連本地的sockect，用sockect與ViewServer進行通訊。

可以看到，dump出來的信息很全，而且是在用戶進程中才有的信息。所以ViewServer會通過mWindowToken中的打印。調用鏈是這樣的：

WMS:viewServerWindowCommand

-> ViewRootImpl:executeCommand

-> ViewDebug:dispatchCommand

-> ViewDebug:dumpViewProperties

打印到目標屬性用了注解+反射

private static void dumpViewProperties(Context context,Object view,

BufferedWriter out,String prefix) throws IOException {

if (view == null) {

out.write(prefix + "=4,null ");

return;

}

Class> klass = view.getClass();

do {

exportFields(context,view,out,klass,prefix);

exportMethods(context,prefix);

klass = klass.getSuperclass();

} while (klass != Object.class);

}

在每個view中，如果允許dump的話，就加入注解,比如說

@ViewDebug.ExportedProperty(flagMapping = {

@ViewDebug.FlagToString(mask = SYSTEM_UI_FLAG_LOW_PROFILE,

equals = SYSTEM_UI_FLAG_LOW_PROFILE,

name = "SYSTEM_UI_FLAG_LOW_PROFILE",

outputIf = true),

@ViewDebug.FlagToString(mask = SYSTEM_UI_FLAG_HIDE_NAVIGATION,

equals = SYSTEM_UI_FLAG_HIDE_NAVIGATION,

name = "SYSTEM_UI_FLAG_HIDE_NAVIGATION",

@ViewDebug.FlagToString(mask = PUBLIC_STATUS_BAR_VISIBILITY_MASK,

equals = SYSTEM_UI_FLAG_VISIBLE,

name = "SYSTEM_UI_FLAG_VISIBLE",

outputIf = true)

},formatToHexString = true)

int mSystemUiVisibility;

hierarchyviewer小結：

hierarchyviewer 就是通過在wms中開啟一個viewServer，把server的端口forward到本地端口上。socket連本地端口，發送 DUMP $(WindowHash),通過binder調用到具體的某個view，遞歸打印出有加注解的信息，回信息給socket，hierarchyviewer(gui)再對信息進行整理，獲得絕對位置等信息。

小結

本文提供了四種獲取當前顯示屏幕的文字信息的方法，簡單分析了下原理。

接下來準備拓展下uiautomator，就可以支持文字查找與點擊了。實現看這篇文章:

Android抓取文字、文字位置的實現

如果對精確度有高要求，對速度有高要求，可以考慮下移植下hierarchyviewer部分的實現。這部分完全重寫需要不少時間，暫時偷個懶吧，不實現了：)

尾聲

用文字匹配搶微信紅包，查資料的時候發現網上不少這樣的文章。下面只是針對這個場景的個人猜想，沒有實際調研。

攻擊的話，看大多數的文章都提到用Accessibility,其實就是用第三種方法(uiautomator)。可以考慮用第四種方法(hierarchyviewer)。就不會受到Accessibility的限制了。

防守的話，新消息那一欄，用自定義view、非規則view，那么有新紅包這個文字信息就抓不到。

掃碼關注，實時互動

總結

以上是生活随笔為你收集整理的抓取android ui原理,Android抓取文字、文字位置的分析的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。