How to use WWWGrep to check the security of your website elements

About WWWGrep

WWWGrep is a tool for HTML security. It implements its functions based on the fast search "grepping" mechanism, checks HTML elements by type, and allows single, multiple, or recursive searches. The Header name and value can also be searched recursively in this way.

Function introduction

Use the recursion option to search the target site for an input field named "username" or "password" to quickly locate the login page.

Quickly check the Header for usage of a specific technology.

Quickly locate cookies and JWT tokens by searching the response Header.

Use with proxy tools to quickly and automatically perform recursion through a set of links.

Find all input receivers on the page (or site) by searching for input fields and parameter processing symbols.

Find all developer comments on the page to identify the commented out code (or to-do).

Quickly find vulnerable JavaScript code in web pages.

Identify the API token and access key present in the page code.

Quickly test whether multiple sites under management use vulnerable code.

Quickly test whether multiple sites under management use vulnerable frameworks / technologies.

Look for sites that may share a common code base to determine the impact of defects / vulnerabilities.

Find sites that share a public authentication token (Header authentication token).

Other features

Tool installation

Researchers can use the following commands to clone the source code of the project locally and install the relevant dependent components:

git clone https://github.com/OWASP/wwwgrep.git
pip3 install -r requirements.txt
python3 wwwgrep.py <arguments and parameters>

Dependent components (pip3 install -r requirements.txt)

 - Python 3.5+
 - BeautifulSoup 4
 - UrlLib.parse
 - requests_html
 - argparse
 - requests
 - re
 - os.path

Command line options

wwwgrep.py [target/file] [search_string] [search params/criteria/recursion etc]
Search Inputs
search_string Specify the string to search, or specify '' for all objects of the type specified in the search parameters
 -t --target Specify a single URL As the target of the search
 -f --file Specify the that contains the to search for URL List of files
Recursion
 -rr --recurse-root take URL Recursion is restricted to the domain provided in the target
 -ra --recurse-any Allow recursive expansion beyond the target domain
Matching Criteria
 -i --ignore-case   Perform case insensitive matching (default is by case)
 -d --dedupe       Allow duplicate results per page (the default is de duplicated results)
 -r --no-redirects   Redirection is not allowed (default is to allow redirection)
 -b --no-base-url   Omit matching from output URL(Include by default URL)
 -x --regex        Regular expression matches are allowed (the search string is treated as a regular expression, and the default value is off)
 -e --separator  Specify and output specifiers (default:)
 -j --java-render   Open page objects and text JavaScript Rendering (off by default)
 -p --linked-js-on  Open link (script) src Mark) Java Search function of script (off by default)
Request Parameters 
 -ps --https-proxy With“ https://The < IP >: < port > "format specifies the proxy for the HTTPS protocol
 -pp --http-proxy With“ https://The < IP >: < port > "format specifies the proxy of the HTTP protocol
 -hu --user-agent Specifies the string used as the user agent in the request
 -ha --auth-header Specify which requests to Header A hosted token or other authentication string used in
Search Parameters
 -s --all       On all pages HTML Terms that match the search in the script
 -sr --relative       Search matching relative URL Page links
 -sa --absolute   Search matching absolute URL Page links
 -si --input-fields   Search the page for matching input fields
 -ss --scripts       Search for script tags that match the search specification
 -st --text          The visible text on the search page that matches the search specification
 -sc --comments     Comments on the search page that match the search specification
 -sm --meta          Search page metadata for matches to the search specification
 -sf --hidden        Search hidden fields for specific matches to the search specification
 -sh --header-name   Search response Header To find a specific match to the search specification
 -sv --header-value   Search response Header Value to find a specific match to the search specification

Tool usage example

Recursively find all input fields named login on the site. Matching is not case sensitive:

wwwgrep.py -t https://www.target.com -i -si "login" -rr

Look for all notes that contain the word "to do" on all pages of the website:

wwwgrep.py -t https://www.target.com -i -sc "to do" -rr

Find all comments on a specific page:

wwwgrep.py -t https://www.target.com/some_page -i -sc ""

Use site recursion to find all hidden fields in the list of web applications contained in the input.txt file:

wwwgrep.py -f input.txt -sf "" -rr

Project address

https://github.com/OWASP/wwwgrep

Posted on Mon, 08 Nov 2021 09:12:01 -0500 by Enlightened

Programmer Help