catalogue
1. First look at the dynamic rendering
1. Determine the screen LED resolution for product online deployment
2, Overall architecture design
3, Implementation of key coding for crawler acquisition
1. Determine crawler target URL
4, Key coding implementation of data visualization
As mentioned above, I recently received suggestions from many small partners. If real data analysis and calculation are adopted, the data will be displayed on the large screen, which will be closer to the actual work scene of small partners and can be applied in work soon. Therefore, according to the needs of small partners, this [bilibili ranking - data visualization large screen solution] of data visualization + crawler was born.
Don't say much, start sharing dry goods, welcome to discuss! QQ wechat same No.: 6550523
❤️ Effect display ❤️
1. First look at the dynamic rendering
2. Rich theme styles
I Determine the demand scheme
1. Determine the screen LED resolution for product online deployment
1280px*768px, F11 full screen without scroll bar; Other resolution screens can be displayed adaptively.
2. Deployment mode
- Based on installation free executable program: support various mainstream operating systems such as Windows, Linux and Mac; Copy the executable program exe to the server without other environment dependence;
- Viewing mode: it can not only directly view the program interface on the server, but also open and play remotely using the browser. It supports mainstream browsers such as Chrome browser and 360 browser.
2, Overall architecture design
- The front end is designed based on Echarts open source library and uses WebStorm editor;
- The back-end is implemented based on Python Web and uses pychart editor;
- Data transmission format: JSON;
- Data source type: this case adopts python request to collect real-time data. In the actual development requirements, it supports customizing HTTP API interface mode or other various types of databases, such as PostgreSQL, MySQL, Oracle, Microsoft SQL Server, SQLite, Excel table, etc.
- Data update method: in order to display data, this case adopts the method of regular pull. In the actual development requirements, the back-end data is updated in real time and pushed to the front-end display in real time;
3, Implementation of key coding for crawler acquisition
1. Determine crawler target URL
The target of this collection is bilibili real-time hotspot data. The url address is: bilibili is a well-known video barrage website in China. It has the most timely animation, the best ACG atmosphere and the most creative Up master. You can find a lot of joy here.
https://www.bilibili.com/v/popular/rank/ent The API address to be called is: https://api.bilibili.com/x/web-interface/ranking/v2?type=all
2. Crawl key code
def scrapy(url):
headers = {
"Content-Type": 'application/json;charset=utf-8',
# "cookie": cookie,
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
}
result_obj = {}
try:
response = requests.get(url=url, headers=headers, verify=False)
result_obj = json.loads(response.text)
except Exception as e:
print(e)
if 0 == result_obj['code']:
return result_obj["data"]["list"]
else:
result_obj['code'] = 'error'
return result_obj
4, Key coding implementation of data visualization
1. Front end html code
<div class="container_fluid">
<div class="row_fluid" id="vue_app">
<div style="padding:0px; height:10%; " class="col-xs-12 col-md-12">
<dv-decoration-4 :reverse="true" style="height:20%;">
</dv-decoration-4>
<h3 style="color:#cba871;text-align:center;height:80%;" id="container_h">container_h</h3>
</div>
<div style="padding: 0px;" class="col-xs-12 col-md-4">
<div style="padding: 0px; height:60%;" class="col-xs-12 col-md-12">
<dv-border-box-2 style="padding: 12px; " :color="['#cba871', '#25201f']">
<h6 style=" color:#cba871; text-align: center; "> - bilibli video playback (iframe mode) -< / H6 >
<div style="padding:5%; height:90%; " id="container_2"><iframe
src="//player.bilibili.com/player.html?aid=891284092&bvid=BV1eP4y1b7Pt&cid=432844457&page=1"
scrolling="no" border="0" frameborder="no" framespacing="0" allowfullscreen="true"
width="100%" height="100%"> </iframe></div>
</dv-border-box-2>
</div>
</div>
<div style="padding:0px;" class="col-xs-12 col-md-4">
<div style="padding:0px; height:60%;" class="col-xs-12 col-md-12">
<dv-border-box-2 style="padding: 12px; " :color="['#cba871', '#25201f']">
<h6 style=" color:#Cba871; text align: Center; "> - regional distribution - < / H6 >
<div style="padding:5%; height:90%;" id="container_3">container_3</div>
</dv-border-box-2>
</div>
</div>
<div style="padding: 0px;" class="col-xs-12 col-md-4">
<div style="padding: 0px; height:30%;" class="col-xs-12 col-md-12">
<dv-border-box-2 style="padding: 12px; " :color="['#cba871', '#25201f']">
<h6 style=" color:#cba871; text-align: center; height:3%; "> - user age & gender distribution - < / H6 >
<div style="padding: 0px; " class="col-xs-12 col-md-6">
<div style="padding:5%; height:90%;" id="container_4_1">container_4_1</div>
</div>
<div style="padding: 0px; " class="col-xs-12 col-md-1">
<div style="width: 0.3px; align:center; height: 80%; background: #cba871;"></div>
</div>
<div style="padding: 0px; " class="col-xs-12 col-md-5">
<div style="padding:5%; height:90%;" id="container_4_2">container_4_2</div>
</div>
</dv-border-box-2>
</div>
<div style="padding: 0px; height:30%;" class="col-xs-12 col-md-12">
<dv-border-box-2 style="padding: 12px; " :color="['#cba871', '#25201f']">
<h6 style=" color:#Cba871; text align: Center; height: 3%; "> - access source & member level - < / H6 >
<div style="padding: 0px; " class="col-xs-12 col-md-6">
<div style="padding:5%; height:90%;" id="container_5_1">container_5_1</div>
</div>
<div style="padding: 0px; " class="col-xs-12 col-md-1">
<div style="width: 0.3px; align: center; height: 80%; background: #cba871;"></div>
</div>
<div style="padding: 0px; " class="col-xs-12 col-md-5">
<div style="padding:5%; height:90%;" id="container_5_2">container_5_2</div>
</div>
</dv-border-box-2>
</div>
</div>
<div style="padding:0px;" class="col-xs-12 col-md-12">
<div style="padding:0px; height:30%;" class="col-xs-12 col-md-12">
<dv-border-box-2 style="padding: 12px; " :color="['#cba871', '#25201f']">
<h6 style="color:#cba871; text-align: center; "> - Hot List Ranking of the whole station (python crawls bilibli) -< / H6 >
<div style=" padding:1%; height:90%; " id="container_6">container_6</div>
</dv-border-box-2>
</div>
</div>
</div>
</div>2. Front end JS code
var idContainer_6 = "container_6";
var chartDom = document.getElementById(idContainer_6);
var myChart = echarts.init(chartDom, window.gTheme);
var option;
option = {
grid: {
left: "1%",
right: "1%",
bottom: "15%",
top: "1%",
containLabel: true,
},
tooltip: {
trigger: "axis",
axisPointer: {
type: "none",
},
formatter: function (params) {
return params[0].name + ": " + params[0].value;
},
},
dataZoom: [
{
type: "slider",
xAxisIndex: 0,
start: 0,
end: 20,
bottom: "-5%"
},
],
xAxis: {
data: [],
axisTick: { show: false },
axisLine: { show: false },
axisLabel: {
color: "rgba(255,255,255,.8)",
},
},
yAxis: {
splitLine: { show: false },
axisTick: { show: false },
axisLine: { show: false },
axisLabel: { show: false },
},
// color: ["#cba871"],
series: [
{
type: "pictorialBar",
barCategoryGap: "-100%",
label: {
normal: {
show: true,
position: "top",
textStyle: {
color: ["#cba871"],
},
},
},
symbol: "path://M0,10 L10,10 C5.5,10 5.5,5 5,0 C4.5,5 4.5,10 0,10 z",
itemStyle: {
opacity: 1,
},
emphasis: {
itemStyle: {
opacity: 1,
},
},
data: [],
z: 10,
},
],
};
function asyncData_6() {
$.getJSON("json/6.json").done(function (data) {
var myChart = echarts.init(document.getElementById(idContainer_6));
myChart.setOption(data);
}); //end $.getJSON
}
window.addEventListener("resize", function () {
myChart.resize();
});
myChart.setOption(option);
asyncData_6();
3. Backend python server code
# -*- coding:utf-8 -*-
import json
import scrapyBilibili as scrapyBilibili
from http.server import HTTPServer, SimpleHTTPRequestHandler, ThreadingHTTPServer
ip = "localhost" # Listening IP, configuration item
port = 8819 # Listening port, configuration item
class MyRequestHandler(SimpleHTTPRequestHandler):
protocol_version = "HTTP/1.0"
server_version = "PSHS/0.1"
sys_version = "Python/3.7.x"
target = "./" # Listening directory, configuration item
def do_GET(self):
url = ""
data = []
# Handling dynamic requests from clients
if self.path.find("/json/6.json") >= 0:
data = scrapyBilibili.filter(scrapyBilibili.scrapy("https://api.bilibili.com/x/web-interface/ranking/v2?type=all"))
else:
SimpleHTTPRequestHandler.do_GET(self)
return
# Response http header
self.send_response(200)
self.send_header("Content-type", "json")
self.end_headers()
# Response http response
rspstr = json.dumps(data)
self.wfile.write(rspstr.encode("utf-8"))
# def do_GET(self):
# if self.path.find("/json/") > 0:
# print(self.path)
# req = {"success": "true"}
# self.send_response(200)
# self.send_header("Content-type", "json")
# self.end_headers()
# with open(self.path, 'r', encoding="utf-8") as f:
# data = json.load(f)
# rspstr = json.dumps(data)
# self.wfile.write(rspstr.encode("utf-8"))
# else:
# SimpleHTTPRequestHandler.do_GET(self);
def do_POST(self):
data = self.rfile.read(int(self.headers["content-length"]))
data = json.loads(data)
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
rspstr = json.dumps(data, ensure_ascii=False)
self.wfile.write(rspstr.encode("utf-8"))
def HttpServer():
try:
server = HTTPServer((ip, port), MyRequestHandler)
listen = "http://%s:%d" % (ip, port)
print("Server listening address: ", listen)
server.serve_forever()
except Exception as e:
print("Exception", e)
server.socket.close()
if __name__ == "__main__":
HttpServer()
5, Online operation
6, Source download
At the end of this sharing, welcome to discuss! QQ wechat same No.: 6550523
7, Highlights summary
Please raise your lovely little hand and poke: ❤️ Data visualization wonderful case summary ❤️_ Little monster blog - CSDN blog