Glide disk cache

Disk cache

Disk cache

  • The implementation through DiskLruCache is similar to the LruCache algorithm. When the cache size will exceed the threshold, the oldest data in the cache will be cleared
    • DiskLruCache is also implemented internally through LinkedhashMap
    • The same is created when glide is generated
  • Here we also analyze taking and saving from two angles

take

Through the analysis of the previous article, we know that when there is no memory cache and active cache, we need to start the thread to obtain data from the disk or network. Let's see the code below

 public <T, Z, R> LoadStatus load(Key signature, int width, int height, DataFetcher<T> fetcher,
            DataLoadProvider<T, Z> loadProvider, Transformation<Z> transformation, ResourceTranscoder<Z, R> transcoder,
            Priority priority, boolean isMemoryCacheable, DiskCacheStrategy diskCacheStrategy, ResourceCallback cb) {
		//1. Build enginejob
        EngineJob engineJob = engineJobFactory.build(key, isMemoryCacheable);
     	//2 create a DecodeJob
        DecodeJob<T, Z, R> decodeJob = new DecodeJob<T, Z, R>(key, width, height, fetcher, loadProvider, transformation,
                transcoder, diskCacheProvider, diskCacheStrategy, priority);
       //3 create EngineRunnable to load pictures from disk or network
     EngineRunnable runnable = new EngineRunnable(engineJob, decodeJob, priority);
        jobs.put(key, engineJob);
        engineJob.addCallback(cb);
     	//4. Call the start method to execute runnable
        engineJob.start(runnable);
        return new LoadStatus(cb, engineJob);
    }
  • EngineJob

    • Manage loaded classes by adding and deleting loaded callbacks and notifying callbacks when loading is complete.

    • Internal maintenance

      • The diskCacheService thread pool is used to perform load tasks

        • diskCacheService = new FifoPriorityThreadPoolExecutor(1);
          
      • The sourceService thread pool is used to perform source data loading, such as obtaining data from the network

        • final int cores = Math.max(1, Runtime.getRuntime().availableProcessors());
          sourceService = new FifoPriorityThreadPoolExecutor(cores);
          
      • Both are initialized and assigned during Glide creation

  • DecodeJob

    • A class responsible for decoding resources from cached data or raw sources and applying transformations and transcoding.
    • Responsible for decoding, converting the original data into corresponding types of pictures
  • EngineRunnable

    • A runnable class, which is responsible for decoding the resources on the background thread in two stages using DecodeJob.
  • By observing the description of the above three objects, we know

    • The engine runnable is executed through the thread pool in the engine job, and decoding is completed through the DecodeJob

    • The real fetch operation is in the EngineRunnable, so we analyze the run method of EngineRunnable

    • Call the start method to execute EngineRunnable

      • public void start(EngineRunnable engineRunnable) {
                this.engineRunnable = engineRunnable;
                future = diskCacheService.submit(engineRunnable);
        }
        
        • You can see that it is completed through diskCacheService. Analyze the Run method below
  • Run

     @Override
        public void run() {
            Resource<?> resource = null;
            // First place
            resource = decode();//Executed by the decode method
            if (resource == null) {
                //Second place
                onLoadFailed(exception);
            } else {
                // Third place
                onLoadComplete(resource);
            }
        }
    
  • decode

        private Resource<?> decode() throws Exception {
            // Determine whether to use disk cache
            if (isDecodingFromCache()) {
                //1. If there is a disk cache, cache and read. If there is no disk cache, the default is to decide whether to cache according to the size and number of pictures. We also start from here
                return decodeFromCache();
            } else {
                // 2. Otherwise, go to the network to request 
                return decodeFromSource();
            }
        }
    
    • You can see that the above image can be loaded in two cases according to the conditions

      • Whether the condition isDecodingFromCache() uses disk cache

        • It depends on whether DiskCacheStrategy.NONE is used when using the API, that is, no pictures are cached, that is, disk caching is disabled. If this value is not set, it is obtained from disk cache by default

        •     private boolean isDecodingFromCache() {
                  // Judgment stage 
                  return stage == Stage.CACHE;
              }
          
      1. If the disk cache is not disabled, load it from the disk cache first - if the disk is not available on the network
        • At this point, call onLoadFailed to know how to execute the network request
      2. If disk caching is disabled, it is loaded directly from the network

Get data from disk cache decodeFromCache()

  • Through the decode method, we can get

    • When using disk cache, call ecodeFromCache(); Get data from disk

    • private Resource<?> decodeFromCache() throws Exception {
              Resource<?> result = null;
              try {
                  //1 get the converted picture resources first
                  result = decodeJob.decodeResultFromCache();
              } catch (Exception e) {
                  if (Log.isLoggable(TAG, Log.DEBUG)) {
                      Log.d(TAG, "Exception decoding result from cache: " + e);
                  }
              }
      
              if (result == null) {
                  //2. If not, get the cache of the original data
                  result = decodeJob.decodeSourceFromCache();
              }
              return result;
          }
      //1 get the transcoded picture resource decodeResultFromCache first
       public Resource<Z> decodeResultFromCache() throws Exception {
      		//...
              Resource<T> transformed = loadFromCache(resultKey);
      		//...
              Resource<Z> result = transcode(transformed);
              return result;
          }
      //2. If it cannot be obtained, the original data transcodesourcefromcache will be obtained
      public Resource<Z> decodeSourceFromCache() throws Exception {
             	//...
              Resource<T> decoded = loadFromCache(resultKey.getOriginalKey());
           	//...
              return transformEncodeAndTranscode(decoded);
          }
      // Both of them call loadFromCache to obtain the original data or transcoding data of disk cache
       private Resource<T> loadFromCache(Key key) throws IOException {
           	//1. Obtain the diskCache through the diskCacheprovider, and obtain the resources corresponding to the key through the diskCache
           	//The diskCacheProvider is generated when the Engine is created. The diskCacheFactory created in GlideBuilder during Glide creation is the parameter
              File cacheFile = diskCacheProvider.getDiskCache().get(key);
              if (cacheFile == null) {
                  return null;
              }
      
              Resource<T> result = null;
              try {
                  //2 get the decoder to decode the cache through the loadloader
                  result = loadProvider.getCacheDecoder().decode(cacheFile, width, height);
              } finally {
                  if (result == null) {
                      diskCacheProvider.getDiskCache().delete(key);
                  }
              }
              return result;
          }
      
      • Through the above code

        • First, get the picture cache converted from the original data

        • If not, get the cache of the original data

        • The cache gets the interpretation at this time

          • Then the code returns to the run method and starts executing the second place

            • If the cache is not empty, call onLoadComplete to notify the main thread that the data loading is complete

            • If the cache is onLoadFailed

              •     private void onLoadFailed(Exception e) {        // When loading fails, the stage changes to obtain and enable network loading if (isdecodingfromcache()) {stage = stage. Source; / / enable network loading manager. Submitforsource (this);} else {manager. Onexception (E);}}
                
              •     @Override    public void submitForSource(EngineRunnable runnable) {       	// Execute the network request through the sourceservice thread pool. future = sourceService.submit(runnable);}
                
              • At this time, the network request to obtain resources from the network has been started. Here, the resources obtained from the network are cached in the disk cache

Get data from the network decodeFromSource()

From the above

  • If disk addition is not disabled, call manager.submitForSource(this) when disk loading fails;
    • The manager is the EngineJob and calls sourceService.submit(runnable);
      • Execute runnable through sourceService thread pool
    • It is equivalent to executing the run method again. At this time, the stage = stage. Source, and the return value of isdecodingfromcache is false. Execute decodeFromSource() to load from the network
  • If disk caching is disabled, the return value of isDecodingFromCache is false. Execute decodeFromSource() to load from the network
  • private Resource<?> decodeFromSource() throws Exception {        return decodeJob.decodeFromSource(); }//decodeFromSource()public Resource<Z> decodeFromSource() throws Exception {     	//  1 call decodeSource() to get resource < T > decoded = decodeSource(); 	//  2 transcode the resource back to return transformencodeandtranscode (decoded);} / / decodeSource() private resource < T > decodeSource() throws exception {resource < T > decoded = null; try {/ / 1 through fetcher.loadData. The priority here is the previous request final a data = fetcher.loadData (priority) ; / / 2 decoded = decodefromsourcedata (data);} finally {fetcher. Cleanup();} return decoded;}
    
  • Through the above code, we find that

    • fetcher.loadData(priority); To obtain network resources

      • fetcher is an object of DataFetcher type, which was created by ModelLoader in onSizeReady above
        • DataFetcher is an interface with multiple implementation classes. These implementation classes are datafetchers that load data from specified data sources
          • A new instance is created by ModelLoader to specify the data source to load
          • For example, if we use String, the corresponding implementation class is HttpUrlFetcher
    • priority is request, which is not used here

    • We pass it as Url, so the loadData is the loadData of HttpUrlFetcher

      • public InputStream loadData(Priority priority) throws Exception {    return loadDataWithRedirects(glideUrl.toURL(), 0 /*redirects*/, null /*lastUrl*/, glideUrl.getHeaders());}// Get inputstreamprivate inputStream loaddatawithredirects (url, int redirects, url lasturl, map < string, string > headers) throws IOException{ 	// Get inputStream urlConnection through HttpURLConnection = connectionfactory. Build (url); / /... Omit setting parameters for urlConnection, partial timeout, etc. urlConnection.connect(); 	//  Get the status final int statusCode = urlConnection.getResponseCode(); / / judge that the status is 200, and return if (statuscode / 100 = = 2) {return getstreamforsuccessfulrequest (urlConnection);} else if (statuscode / 100 = = 3) {/ / if the status is 300, get the redirected url string for redirection. Redirecturlstring = urlConnection.getheaderfield ("location"); / / if the redirected url is empty, throw an exception if (textutils. Isempty (redirecturlstring)) {throw new IOException ("received empty or null redirect url");} //If the redirect url is not empty, call loadDataWithRedirects() again. The parameters are redirect url, redirecturl = new url (url, redirecturlstring); return loaddatawithredirects (redirecturl, redirects + 1, url, headers);}}
        
  • At this point, the network request ends

Network request summary

  • Call LoadData through the data loader DataFetcher corresponding to the data source
    • Here, we take Url as an example. The corresponding DataFetcher is HttpUrlFetcher. LoadData obtains data internally through loadDataWithRedirects
    • Internally, HttpURLConnection is built through the url parameter set for Glide
      • Get the states by connecting to the urlConnection object
        • Determine whether statescode is 200
          • If yes, get the inputStream in urlConnection and return
        • Determine whether statescode is 300
          • If yes, you need to redirect, get the redirect url, and judge whether the url is empty
            • If yes, an exception is thrown
            • Otherwise, build a new Url according to the redirect Url and call the method loadDataWithRedirects again to get the data
      • So far, when statescode=200, the data request ends, and the code is executed to the code at 2 in the decodeSource. decodeFromSourceData decodes the source data

Decoding & storing raw data and decoder

So far, when statescode=200, the data request ends, and the code is executed to the code at 2 in the decodeSource. decodeFromSourceData decodes the source data

    private Resource<T> decodeFromSourceData(A data) throws IOException {        final Resource<T> decoded;        // 1 if disk cache is used, the original data and decoder are cached. If (diskcachestrategy. Cachesource()) {decoded = cacheanddecodesourcedata (data);} else {/ / 2 if disk cache is not applicable, the original data is decoded according to the decoder. decoded = loadProvider.getSourceDecoder().decode(data, width, height) } / / returns the decoded resource return decoded;}

Location 1 if disk cache is used, the original data and decoder are cached

If disk caching is used, the original data is decoded to the cache

 private Resource<T> cacheAndDecodeSourceData(A data) throws IOException {       	// Create SourceWriter internal maintenance decoder and original data SourceWriter < a > writer = new SourceWriter < a > (loadprovider. Getsourceencoder(), data); 	//  Cache the writer to the disk cache diskCacheProvider.getDiskCache().put(resultKey.getOriginalKey(), writer)// Get cache assignment to result resource < T > result = loadfromcache (resultkey. Getoriginalkey())// return result;}
  • loadProvider inherits DataLoadProvider, which is an interface

    • A resource loader that provides the necessary encoders and decoders to decode a particular type of resource from a particular type of data.

    • Create according to when generating GenericRequestBuilder

      • When calling the constructor of super in the constructor of DrawableTypeRequest, call buildProvider to generate

        • buildProvider(glide, streamModelLoader, fileDescriptorModelLoader, GifBitmapWrapper.class,GlideDrawable.class, null)
          

Position 2 if disk cache is prohibited, decoding returns

loadProvider.getSourceDecoder().decode(data, width, height);
  • Call loadProvider.getSourceDecoder() to transfer the call to StreamBitmapDataLoadProvider. The actual call is streambitmapdecoder (because we take url as an example here). The called decode method is as follows

    • public Resource<Bitmap> decode(InputStream source, int width, int height) {    // Decode it into bitmap. Don't read the specific decoding. If you are interested, analyze Bitmap bitmap = downsampler.decode(source, bitmapPool, width, height, decodeFormat)// Encapsulate bitmap and bitmap pool into BitmapResource and return BitmapResource.obtain(bitmap, bitmapPool);}
      
    • At this time, the decoding is completed and the execution return returns resourcedecodeSource(). The execution completion code is specified. The code transformEncodeAndTranscode(decoded) at position 2 in the decodeFromSource transcodes the data source requested by the network and saves it to the disk cache at the same time

Decoding memory summary

  • If disk caching is used, the original data and decoder are cached
  • If disk cache is not used, the original data is decoded and returned according to the decoder

Conversion & save conversion data

transformEncodeAndTranscode(decoded)

private Resource<Z> transformEncodeAndTranscode(Resource<T> decoded) {        long startTime = LogTime.getLogTime();    	// 1. Transform data resource < T > transformed = transformed (decoded); if (Log.isLoggable(TAG, Log.VERBOSE)) {            logWithTimeAndKey("Transformed resource from source", startTime);        } 		//  2 store the transformed data in the local writetransformed tocache (transformed)// 3 transcode the data into the corresponding picture type resource and return it. For example, convert bitmap into drawable resource < z > result = transcode (transformed); if (Log.isLoggable(TAG, Log.VERBOSE)) {            logWithTimeAndKey("Transcoded transformed from source", startTime);        }        return result;    }

Position 1 code data conversion transform(decoded);

  • It implements the bitmapTransform() method of loading image settings in Glide, and sets the functions we need, such as blurred images and rounded images

    • private Resource<T> transform(Resource<T> decoded) { 	// Transform resource < T > transformed = transformation.transForm (decoded, width, height); return transformed;}
      
      • Transformation is an interface used to perform arbitrary transformation on resources. Its implementation classes are as follows

        • BitmapTransformation

        • GifBitmapWrapperTransformation

        • GifDrawableTransformation

        • MultiTransformation<T>

        • UnitTransformation<T>

        • Take for example

          •     public final Resource<Bitmap> transform(Resource<Bitmap> resource, int outWidth, int outHeight) {       	//Get Bitmap bitmap totransform = resource. Get(); int targetWidth = outWidth == Target.SIZE_ ORIGINAL ?  toTransform.getWidth() : outWidth;         int targetHeight = outHeight == Target.SIZE_ ORIGINAL ?  toTransform.getHeight() : outHeight;        // Bitmap transformed = transform(bitmapPool, toTransform, targetWidth, targetHeight)// Create a resource < bitmap > result assignment and return the final resource < bitmap > result// Judge that if the data before conversion is the same as the data after conversion, the unconverted data will be returned if (totransform. Equals (transformed)) {result = resource;} else {/ / if it is different, the converted data will be returned. Result = bitmapresource.obtain (transformed, bitmappool);} return result;}
            

Location 2 code is stored in writetransformed tocache (transformed);

Execute cache action

private void writeTransformedToCache(Resource<T> transformed) {    if (transformed == null || !diskCacheStrategy.cacheResult()) {        return;    }    long startTime = LogTime.getLogTime();    //Build a SourceWriter to encapsulate the encoder and converted data SourceWriter < resource < T > > writer = new SourceWriter < resource < T > > (loadprovider. Getencoder(), transformed)// Store the data returned after conversion into the disk cache. Whether the cached data is converted data or converted data is determined by comparing the hash addresses before and after conversion in the conversion method. If / / if the data before conversion is the same as the data after conversion, store the unconverted data. / / if it is different, store the converted data diskCacheProvider.getDiskCache().put(resultKey, writer);}

Transformation & Survival summary

  • Compare the transcoded data with the transcoded data to determine whether it is the same
    • If the same, the untranslated data is cached
    • Cache transcoding data if different

Process summary

  1. When the decodeSource() network requests data, it calls decodeFromSourceData.

    • decode

    • Judge whether to disable caching. If not, build a SourceWriter to encapsulate and cache the decoder and original data, and take out the data from the cache and return it

    • If caching is disabled, the decoded data will be directly decoded and returned

  2. change

    • When the encoding is completed, the transformEncodeAndTranscode is called to transform the data. After transformation, SourceWriter is built to encapsulate the encoder and the converted data into the cache.
  3. Call transcode(transformed); transcode the data to the corresponding picture resource and return

    • Decoding transcoding process

      • glide cannot load bitmap directly, but it can convert bitmap into drawable or byte array

      • After Glide grabs the data, it will be converted into InputStream. At this time, through the idea of type model conversion, find the decoder that can decode InputStream from the decoding register, including StreamBitmapDecoder and StreamGifDecoder. We know that the most StreamBitmapDecoder can successfully decode the data and become a Bitmap data.

        We know in Glide.with(this).load(url).into(iv_img); that our goal is to obtain a Drawable, that is, transcodeClass is actually Drawable.class. Therefore, through matching search, we can obtain the BitmapDrawableTranscoder transcoder.

Tags: Java Database Cache glide

Posted on Mon, 27 Sep 2021 14:28:51 -0400 by vijayanand