Set up visual data analysis platform redash in 1 hour

Set up visual data analysis platform redash in 1 hour

There are many BI choices, such as Tableau, PowerBI, Superset, Metabase, and redash introduced today. Tableaus are expensive and drag-and-drop. For a data analyst, there is a lot of SQL to write, so consider superset vs. Metabase vs.Reash, superset has been built and used before. Airbnb is beautiful, but it doesn't seem to be very useful, Metabase hasn't been used, PowerBI has known before that it hasn't been studied in depth. The previous company used redash and had a good experience. There are also some limitations in the work, so consider building a simple data analysis locally.Platform to do some simple analysis and monitoring reports, the solution is: Windows 10+Docker+Redash+Python. Python automatically crawls data every day to insert postgresql database, redash writes SQL code to generate reports.

I think this solution would be better than installing a mysql database locally or using Tableau/PowerBI to click a little bit and a half a day there.

Installation steps

# Install docker first
 Download one online Docker for windows perhaps for mac,The latest version is still good, but previous versions were hard to use.
Configure Domestic Mirror: Baidu Search comes soon
{
  "debug": true,
  "registry-mirrors": [
    "https://registry.docker-cn.com"
  ],
  "experimental": true
}


# Download the source from GitHub [you can open GitHub without git and click download and unzip]
git clone https://github.com/getredash/redash.git

# Open cmd/Termina to redash directory
cd redash


# Create a docker-compose.production.yml file that references the YML configuration.
# The git download above also contains a docker-compose.yml file, which is for development purposes and needs to be replaced.
# You can create a new one without touch ing.
touch docker-compose.production.yml

#Create db, install without error
docker-compose -f docker-compose.production.yml run --rm server create_db

#redash runs and there's something installed there as well
docker-compose -f docker-compose.production.yml up

#redash runs in the background
docker-compose -f docker-compose.production.yml up -d

Contents of docker-compose.production.yml file

This configuration REDASH_ADDITIONAL_QUERY_RUNNERS:'redash.query_runner.python'can be used to run Python, but it is python2 and I don't know how to change it to python3. I don't know what to configure these parameters like, just run. It is recommended that the image version not be the latest, download is slow and installation is unsuccessful...

# This is an example configuration for Docker Compose. Make sure to atleast update
# the cookie secret & postgres database password.
#
# Some other recommendations:
# 1. To persist Postgres data, assign it a volume host location.
# 2. Split the worker service to adhoc workers and scheduled queries workers.
version: '2'
services:
  server:
    image: redash/redash:8.0.2.b37747
    command: server
    depends_on:
      - postgres
      - redis
    ports:
      - "5000:5000"
    environment:
      PYTHONUNBUFFERED: 0
      REDASH_LOG_LEVEL: "INFO"
      REDASH_REDIS_URL: "redis://redis:6379/0"
      REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
      REDASH_COOKIE_SECRET: "suibianxieyige"
      REDASH_WEB_WORKERS: 4  
      #mailbox 
      REDASH_MAIL_SERVER: "smtp.exmail.qq.com"
      REDASH_MAIL_PORT: 465
      REDASH_MAIL_USE_TLS: "true"
      REDASH_MAIL_USE_SSL: "true"
      REDASH_MAIL_USERNAME: "xxx@xx.com"
      REDASH_MAIL_PASSWORD: "xxx"
      REDASH_MAIL_DEFAULT_SENDER: "xxx@xx.com"
      REDASH_HOST: "http://192.168.3.110:80"
      REDASH_ADDITIONAL_QUERY_RUNNERS: "redash.query_runner.python"
    restart: always
  worker:
    image: redash/redash:8.0.2.b37747
    command: scheduler
    environment:
      PYTHONUNBUFFERED: 0
      REDASH_LOG_LEVEL: "INFO"
      REDASH_REDIS_URL: "redis://redis:6379/0"
      REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
      REDASH_COOKIE_SECRET: "xxx"
      REDASH_ADDITIONAL_QUERY_RUNNERS: "redash.query_runner.python"
      QUEUES: "queries,scheduled_queries,celery"
      WORKERS_COUNT: 2
    restart: always
  redis:
    image: redis:3.0-alpine
    ports:
     - "6379:6379"
    volumes: 
      - ./data/redis_data:/data
    restart: always
  postgres:
    image: postgres:9.5.6-alpine
    ports:
     - "5432:5432"
    volumes:
      - ./data/postgresql_data:/var/lib/postgresql/data
    restart: always
  nginx:
    image: redash/nginx:latest
    ports:
      - "80:80"
    volumes:
      - ./data/nginx:/etc/nginx
    #depends_on:
    #  - server
    links:
      - server:redash
    restart: always

If the installation is OK, open the browser and enter http://0.0.0.0:5000/You can access it as follows

There is no data source just in, so you need to add one. There are many kinds of data sources that can be added. We add a default database, postgresql, database name and host whatever, in the yaml profile postgresql://postgres@postgres/postgres.


Create a new query, write a sql, and run an analysis platform locally

Dashboard [Fake Data]

Graphs are plotly based, and this library has a python version and is very powerful for drawing.

The query for each chart can set the update time, so this is great. For example, if you want to see the report every morning, you can set the update time at 8 a.m. and turn on the computer at 9 a.m. That's where the data comes from.

Conditionally, you can connect directly to large data platforms or mysql databases or whatever. You can also write your own python code to automatically update your data every day.

It's also nice to think that you can grab house price data and stock market data every day and visualize it yourself. I think this redash is more convenient than Tableau.

So much for now, follow-up:

(1) If you want to change the source code, then you can read the local csv file directly to the database, and you want God to guide you;

(2) You can run python 3 directly instead, run machine learning models, visualize python results directly, etc.

(3) Link neo4j to show social network maps, do fraud analysis, etc.

(4) Explore new usages...

–Over–

Reference link: [Thank you guys for the tutorial]

Installation Tutorial:
http://www.aloneme.cn/2019/07/02/Redash%E8%AF%A6%E7%BB%86Docker%E9%83%A8%E7%BD%B2%E6%95%99%E7%A8%8B/
​www.aloneme.cn/2019/07/02/Redash%E8%AF%A6%E7%BB%86Docker%E9%83%A8%E7%BD%B2%E6%95%99%E7%A8%8B/

No results were queried:

https://github.com/getredash/redash/issues/3761
​github.com/getredash/redash/issues/3761
dockerhub redash:

Docker Hub
​hub.docker.com/r/redash/redash/tags
Selection and Practice Series of Redash as a Data Visualization Scheme

https://medium.com/marsw-redash/bi%E5%B7%A5%E5%85%B7-%E4%BB%A5redash%E7%82%BA%E8%B3%87%E6%96%99%E8%A6%96%E8%A6%BA%E5%8C%96%E6%96%B9%E6%A1%88%E4%B9%8B%E9%81%B8%E6%93%87%E8%88%87%E5%AF%A6%E8%B8%90%E7%B3%BB%E5%88%97-affbf9cb45a6
​medium.com/marsw-redash/bi%E5%B7%A5%E5%85%B7-%E4%BB%A5redash%E7%82%BA%E8%B3%87%E6%96%99%E8%A6%96%E8%A6%BA%E5%8C%96%E6%96%B9%E6%A1%88%E4%B9%8B%E9%81%B8%E6%93%87%E8%88%87%E5%AF%A6%E8%B8%90%E7%B3%BB%E5%88%97-affbf9cb45a6

Team Report System Customization - Redash-based Secondary Development
​www.jianshu.com/p/36ac116f7e0a

There's a lot of stuff on medium and github that you can check out.

redash research
https://cloud.tencent.com/developer/article/1586580
​cloud.tencent.com/developer/article/1586580

Tags: Python Database PostgreSQL

Posted on Mon, 27 Sep 2021 12:26:03 -0400 by Darkness Soul