Reptile requests advanced
requests module advanced
Agent (anti climbing mechanism)
If a high-frequency request is sent to a server in a short time, it will be regarded as an abnormal request and the current IP will be blacklisted
Concept: in a crawler, it refers to a proxy server
Role of proxy server:
Intercept and forward requests and responses
Association between ...
Posted on Thu, 19 Mar 2020 06:45:28 -0400 by shrike202
About the advantages of using the Laravel service container
If the core of the laravel framework is anything, it is undoubtedly a service container. Understanding the concept of service container is very important for us to use laravel. It should be said that understanding the concept of service container is an important condition to distinguish whether to start laravel. Because the whole framework is b ...
Posted on Thu, 19 Mar 2020 04:45:21 -0400 by NArc0t1c
Linux warning: Permison denied thought
In Linux, we often encounter Permisson denied errors.Whether sudo or switching root users continue execution without understanding the logic behind it.Today we'll break and tear Linux's Discretionary Access Control.
Tips: Key terms and content categorization referenced in this article may differ from the standard.Please understand flexibly
Pe ...
Posted on Thu, 19 Mar 2020 03:55:39 -0400 by Joshua F
Self-increasing ID algorithm snowflake
There are no rules for IDs generated using UUID s or GUID s
Snowflake algorithm is implemented by Twitter engineers for incremental but not repetitive ID s
OverviewIn distributed systems, there are scenarios where a globally unique ID is required. In order to prevent ID conflicts, a 36-bit UUID can be used, but it has some drawbacks. First, it ...
Posted on Thu, 19 Mar 2020 03:20:03 -0400 by Tea_J
MAC SonarQube Static Code Check
SonarQube is an open platform for managing code quality to quickly locate potential or obvious errors in your code. The installation, configuration, and use of this tool are described below.(This article refers to others'writings and adds a summary of your own problems)
Dead work;
1. jdk (please Baidu yourself, very easy)
2,sonarqube: http://ww ...
Posted on Thu, 19 Mar 2020 03:08:31 -0400 by kentopolis
Maven+Nexus Environment for Java Packaging Environment Deployment
Maven+Nexus Environment for Java Packaging Environment Deployment
(1) Introduction to the Maven Project
For more information, please go to the official website:
http://maven.apache.org/what-is-maven.htm
We need to know Maven's main introductory information:
1.Maven is the next free open source project belonging to the apache Software Foun ...
Posted on Thu, 19 Mar 2020 03:05:47 -0400 by Marsha
Exploration of asyncPipe Source in Angular
It's the most elegant way to subscribe to observable, not only with a short syntax, but also with automatic unsubscribe.
The async pipeline is used to unpack asynchronous raw data.When it comes to asynchronous data, it's natural to think of observable s and promise s, where async subscribes to them, returns the latest value they've published, ...
Posted on Thu, 19 Mar 2020 01:10:05 -0400 by jackel15
Server Spooky Request Timeout Problem
A few days ago, there were occasional requests for a two-second timeout on the monitor display line.It took a lot of time and some detours to solve this problem.Take notes here.
Pre-phase Analyses
First of all, we need to know about our services:
Our service is a stateless set of front-end servers plus a stateful back-end storage layer.
...
Posted on Thu, 19 Mar 2020 01:04:10 -0400 by kunalk
Python3 Standard Library: mmap Memory Mapping File
1. mmap memory map file
Creating a memory map for a file will use the operating system virtual memory to directly access the data on the file system instead of using regular I/O functions to access the data.Memory mapping typically provides I/O performance because using memory mapping does not require a separate system call for each access or d ...
Posted on Wed, 18 Mar 2020 23:49:29 -0400 by dkjohnson
Learn Java8 Together--CompletableFuture
Synchronous Asynchronous
Computer technology is developing rapidly, both in software and hardware. The CPU of the computer is also being updated. The powerful CPU can undertake more tasks.If the program keeps using synchronous programming, it will waste CPU resources.For example, a CPU has 10 channels. If all programs take one channel, the rema ...
Posted on Wed, 18 Mar 2020 22:58:45 -0400 by jaddy