XIV. Data format conversion of spark log file summarized by Spark SQL

Step 1. Data source Find the log data source of spark. In the directory / root / spark / spark-2.0.2-bin-hadoop 2.7/logs: By reading the file, count the number of data: val masterLog = sc.textFile("file:///root/spark/spark-2.0.2-bin-hadoop2.7/logs/spark-root-org.apache.spark.deploy.master.Master-1-hadoop000.out") val workLog = ...

Posted on Wed, 11 Dec 2019 14:33:55 -0500 by latvaustin

Optimize Case | SQL Optimize in Partition Table Scenario

Reading Guide There is a table partitioned, one partition per day. There is a query on this table, often only query the data of one day in the table, but almost all the data of the whole partition is scanned every time. What can I do to optimize it? 1. Scenes to be optimized There is a large table that generates about 1 million data per day, s ...

Posted on Wed, 11 Dec 2019 01:14:54 -0500 by exec1

Chart design under SSM framework

1. Because the user-defined result set is used in the automatically generated Mapper.xml file (that is, the statistical function only takes a few of the fields, but other fields will still have fields after they are converted to json data, but the value is null, which makes it difficult to select the data.) The effect is not ideal. So define a ...

Posted on Wed, 11 Dec 2019 00:52:53 -0500 by lszanto

java supermarket shopping management system

I. overview 1. Peng GE has written java project supermarket management system project in front of him, Portal 2. received many friends' personal letters to me, and I also appreciated the feedback and communication from the old fellow. The previous project was just a basic knowledge and object-oriented thinking exercise for java, but it didn't i ...

Posted on Tue, 10 Dec 2019 21:42:38 -0500 by YappyDog

C ා expression tree

Preface In the use of EF, we often use xx. Where (P = > p.name = "Zhang San") to query data. This is because the EF framework will turn these C ා codes into Sql statements. The expression tree is mainly used. Today we will learn about the expression tree. Understanding expression tree Func<int,int,int> func = (a, b) => a ...

Posted on Tue, 10 Dec 2019 12:05:18 -0500 by ari_aaron

Is the internal function of the execution plan in ORACLE database an implicit conversion?

In ORACLE database, we will use some SQL statements to find out the problem of implicit conversion SQL. One of the SQL statements circulating on the Internet is as follows. Query whether there is an internal function in the filter prediction field of V $SQL plan:   SELECT      SQL_ID,      PLAN_HASH_VALUE FROM      V$SQL_PLAN X WHERE      X.FI ...

Posted on Tue, 10 Dec 2019 09:14:09 -0500 by GoncaloF

Simple and elegant Python teaches you how to be lazy at work

Preface The text and pictures of this article are from the Internet, only for learning and communication, not for any commercial purpose. The copyright belongs to the original author. If you have any questions, please contact us in time for handling. Author: A prefix PS: if you need Python learning materials, you can click the link below to get ...

Posted on Tue, 10 Dec 2019 06:01:23 -0500 by wallabee

The use of select case when in oracle novice learning

[toc] The use of select case when in oracle novice learning Formatting grammar case when Condition 1 then action1 when Condition 2 then action2 when Condition 3 then action3 when condition N then actionN else action end Example Judge the month SQL> select case substr('20181118',5,2) 2 when '08' then '8yue' 3 when ' ...

Posted on Tue, 10 Dec 2019 02:47:04 -0500 by pokemon

How can ORACLE modify the number of user connections at one time?

Details of oracle connections The default number of oracle connections is 150. When the number of connections at the same time is greater than this number, an error will be reported. The current solution is to increase the number of oracle connections from 150 to 1500. The operation is as follows 1. Problem solving before modification If the nu ...

Posted on Mon, 09 Dec 2019 22:19:48 -0500 by 2levelsabove

postgres's \ d command does not display all user tables

As shown in the following example, there is a table t2 with the same name in different schema of the database, but only one T2 table is queried when using the \ d command. postgres=# select oid,nspname from pg_namespace; oid | nspname -------+-------------------- 99 | pg_toast 11736 | pg_temp_1 11737 | pg_toast_temp_ ...

Posted on Mon, 09 Dec 2019 20:29:05 -0500 by cnagra