requests practice of interface automation preliminary practice

Preface

Previous content mainly understand the initial use of requests, how to send requests, and how to carry request parameters. Then, what aspects of the real world are involved in the actual use process? This practice goal: How to master the operation of requests library to cope with the work environment

1. Format the return value?

The previous article actually showed how to get the return value

res = requests.post(url, data=data)
res.text

As shown in the code above:.text can get the return value of the corresponding url, so let's try to print out what this return value type is

print(type(res.text)) # <class 'str'>

The return value of res.text is "str". In fact, the return value is in json format in its representation. If the return value is of STR type, it is not as expected for the data to be processed as key-value pairs. Thus, in requests, the return value can be.json() to return a json format.

import requests

data = {
    "key":1,
    "value":"Zhang San",
    "value2": "Li Si"
}
res = requests.post("http://httpbin.org/post", data=data)
res.status_code # 200
res.url # http://httpbin.org/post
text = res.text
json_text = res.json()
print(text)
print(json_text)

Let's see how the return value behaves:

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "key": "1", 
    "value": "\u5f20\u4e09", 
    "value2": "\u674e\u56db"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "56", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.25.1", 
    "X-Amzn-Trace-Id": "Root=1-60e6cfb7-0b72133a762d49783c6dac69"
  }, 
  "json": null, 
  "origin": "119.4.64.20", 
  "url": "http://httpbin.org/post"
}

{'args': {}, 'data': '', 'files': {}, 'form': {'key': '1', 'value': 'Zhang San','value2':'Li Si'},'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '56', 'Content-Type': 'application/x-www-form-urlencoded', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.25.1', 'X-Amzn-Trace-Id': 'Root=1-60e6cfb7-0b72133a762d49783c6dac69'}, 'json': None, 'origin': '119.4.64.20', 'url': 'http://httpbin.org/post'}

There seems to be no significant difference in general performance, with or without indentation, just the difference between single quotation marks and double quotation marks. However, there are significant differences in Chinese display.

So let's verify again what we want - I want to process the data, for example, I need value under the form keyword

print(text['form'])
print(json_text['form'])

Obviously, the first line of code will fail

Traceback (most recent call last):
  File "test111111.py", line 143, in <module>
    print(text['form'])
TypeError: string indices must be integers

The second line of code will output value correctly:

{'key': '1', 'value': 'Zhang San', 'value2': 'Li Si'}

2. headers

If you've ever written about crawls, you've experienced blocking access to servers. The initial mechanism for backcrawling is to determine whether you're a user or a program.

Let's do an experiment:

import requests

proxies = {"https":"https://localhost:8888", "http":"http://localhost:8888"}

res = requests.get("http://www.baidu.com", proxies=proxies,verify=False)
print(res)
print(res.headers)

When executing the above code, open fiddler to grab the contents of the request body

At the same address, we use fiddler to grab the content of the request in a browser request

Clearly, the headers content is completely different between actual code requests and browser requests, "User-Agent" directly identifies the way the requests are made for python code. This is an initial anti-crawling technique in the crawl anti-scavenging mechanism. So can you construct a fake headers yourself to execute the requests in code like a construction parameter?

Take User-Agent as an example:

header= {
  "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
}

res = requests.get(url, headers=header)

Sending requests with such request parameters can disguise as a Mozilla browser access. Some sites can bypass the initial anti-crawl mechanism.

Then, in interface testing, there may also be a mechanism within the project that can disguise other parameter values within the header parameter and request them with code.

summary

Focus on the Public Number to learn more

Tags: Python software testing

Posted on Tue, 14 Sep 2021 14:01:03 -0400 by moiseszaragoza