Kylin configures Spark and builds Cube

HDP version: 2.6.4.0Kylin version: 2.5.1 Machine: Three CentOS-7,8G memory In addition to MapReduce, Kylin's computing engine also has a faster SPARK. This article tests the speed at which Spark builds a Cube with Kylin's own example, kylin_sales_cube. 1. Configure Kylin's Spark parameters Before running Spark cubing, it is recommended that yo ...

Posted on Tue, 24 Sep 2019 12:08:38 -0400 by nitediver

Haoop Self-study Diary--3. Haoop HDFS Operation

hadoop self-study Diary - 3.hadoop hdfs operation Setting up the Environment I use a simple windows 7 notebook and use VirtualBox to create a Centos virtual machine to install Hadoop. VirtualBox: 6.0.8 r130520 (Qt5.6.2) CentOS: CentOS Linux release 7.6.1810 (Core) jdk: 1.8.0_202 hadoop: 2.6.5 Clus ...

Posted on Thu, 25 Jul 2019 03:12:49 -0400 by Tory

Haoop Self-study Diary--2. Haoop Cluster Environment Construction

Haoop Self-study Diary --- 2. Haoop Cluster Environment Construction Setting up the Environment I use a simple windows 7 notebook and use VirtualBox to create a Centos virtual machine to install Hadoop. VirtualBox: 6.0.8 r130520 (Qt5.6.2) CentOS: CentOS Linux release 7.6.1810 (Core) jdk: 1.8.0_202 h ...

Posted on Wed, 24 Jul 2019 07:23:53 -0400 by PHPilliterate

centos7 installation Hadoop+hbase+hive steps

centos7 installation Hadoop+hbase+hive steps I. IP, DNS, Host Name linux static IP, DNS, host name configuration Two, Hadoop #### 1. IP allocation #### 2. Install jdk8 (four) yum list java* yum install -y java-1.8.0-openjdk-devel.x86_64 The default jre jdk installation path is under / usr/lib/jvm #### 3. Configure jdk environment variables (f ...

Posted on Thu, 21 Mar 2019 14:06:52 -0400 by drkstr

Secondary NameNode directory structure, nameNode fault recovery

Secondary NameNode directory structure, nameNode failure recovery 1. Secondary NameNode directory structure Secondary NameNode is an auxiliary daemon used to monitor the status of HDFS, which takes snapshots of HDFS metadata at regular intervals. See the Second aryNameNode directory structure in / opt/module/hadoop-2.7.2/dat ...

Posted on Tue, 29 Jan 2019 02:51:15 -0500 by Petrushka

spark Lost executor on YARN

The execution script appears: 15/07/30 10:18:13 ERROR cluster.YarnScheduler: Lost executor 8 on myhost1.com: remote Rpc client disassociated 15/07/30 10:18:13 ERROR cluster.YarnScheduler: Lost executor 6 on myhost2.com: remote Rpc client disassociated The reason for this is that yarn resources are not enough. Increasing resources ...

Posted on Sun, 06 Jan 2019 23:21:09 -0500 by jmansa