Programmer Help

I. The significance of data compression in hadoop 1. Basic overview Compression technology can reduce the number of reading and writing sections of the underlying hdfs. And it can reduce the network bandwidth resources occupied in the process of data transmission and reduce the disk space occupied. In MapReduce, both the shuffle and merge proc ...

Recently, encrypted transmission of important data is needed in the project. We found a large number of encryption methods on the Internet and finally adopted AES encryption. 1, java terminal package com.zk.web.util; /** * AES 128bit Encryption and decryption tool class * @author dufy */ im ...

When the job is submitted to yarn, it will start to schedule and run the map task. Here we start to explain the source code analysis of map input.The entry of a map task is the run() method in MapTask.class 1. First look at the MapTask.run() method MapTask.class //---------------------------------MapTask.java public void run(JobConf job, TaskU ...

This article is based on the previous one.< Getting started with Spring Boot (6): integrating treetable and zTree to implement tree chart >, this article mainly introduces the spring boot integration swagger2. About swagger, Google. I have related video materials purchased on the Internet, and can call me if necessary. 1. Introduce relat ...

There are many video codecs in the machine, such as h264/hevc/h263/mpeg4. Wait. But if you have a new standard encoder, how do you need to integrate it? Read this article and you will find the answer. 1. integration of software encoders: Before integration, we need to understand the loading process ...

I. Realization Principle 1. Using md5 algorithm, a 32-bit signature string is generated for a long web site, which is divided into four segments, each with eight characters. 2. For each of the four cycles, 8 characters are taken as hexadecimal strings and bits and operations of 0x3fffffff(30-bit-1), and more than 30 bits are ignored. It's use ...

1.libx264 is a free H.264 encoding library. It is part of x264 project. It is widely used. ffmpeg's H.264 implementation is libx264. ffmpeg has 264 decode and no encoding. X264 is needed. Go to the official website to download a libx264 compressed file.I used x264-snapshot-20171212-2245.x264 official we ...

1 Overview Compression strategies and principles 2 MR supported compression coding Compressed format hadoop comes with itself. algorithm File extension Is it separable Whether the original program needs to be modified after changing to compressed format DEFLATE Yes, direct use DEFLATE .deflate no As with text processing, no modification ...

Preface I've seen that most companies have their own tool classes encapsulated, and many people have implemented the method again, not only the code quality is not guaranteed, but also the performance and concurrency will be problematic; anyway, I generally do not use the tool classes encapsulated by companies, such as apache. commons, springut ...

The following is based on ffmpeg 2.6.9: Several key structures: AVFormatContext contains a stream information structure AVCodecContext decoder context AVCodec Decoder SwsContext Pixel Information Conversion Context AVPacket contains information about a frame AVFrame Parses a Frame Process summary: av ...