Fast-rcnn-tensorflow-python version 2.7-smallcorgi training own dataset practice notes

Fast-rcnn-tensorflow-python version 2.7-smallcorgi training own dataset actual combat notes (2)

reference material

https://github.com/DexterInd/GoPiGo/issues/210

https://stackoverflow.com/questions/14219092/bash-script-and-bin-bashm-bad-interpreter-no-such-file-or-directory

xuforeverloveCSDN blog

https://github.com/smallcorgi/Faster-RCNN_TF

Training PASCAL VOC 2007 data set

  • 1. Download datasets

    Here you can use the following command to download data from the official, but if you can't climb the wall, it may be very slow, so here's my baidu cloud share Link: https://pan.baidu.com/s/1L5sA1LmhXofIuYvUmlb7Gw
    Extraction code: inds

    wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
    wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
    wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar
    

    If you use Baidu cloud to download, there should be 4 files in it. In addition to the 3 tar files downloaded by using the command, there should be another VGG_imagenet.npy File, no doubt. Another file needs to be used in the next step. If you download using the command, you can go to the official download address Google Drive or Dropbox Download the file

  • 2. Extract and place datasets and related files

    First, enter the directory of the file you downloaded and open the terminal. Enter the following command to extract 3 tar files

    tar xvf VOCtrainval_06-Nov-2007.tar
    tar xvf VOCtest_06-Nov-2007.tar
    tar xvf VOCdevkit_08-Jun-2007.tar
    

    The extracted file will generate a VOCdevkit folder in this directory, which should contain the following files:

    [failed to transfer and save the pictures outside the chain. The source station may have anti-theft chain mechanism. It is recommended to save the pictures and upload them directly (img-9n6ywrj5-1593142595437) (E: \ project_ 1\faster-RCNN\mdimage\ vocdevkit.png ]

    The next steps are different from the official ones, but in principle they are the same. Here we do not link files. We need to rename the above VOCdevkit folder as VOCdevkit 2007 and put it on frcnn_ Under the root / data path, create a new folder named pretrain_model and VGG_image.npy The files are placed in this folder. So now you frcnn_ Under root / data, it should be as follows:

    [failed to transfer and store the pictures in the external link. The source station may have anti-theft chain mechanism. It is recommended to save the pictures and upload them directly (img-emkwzyat-1593142595442) (E: \ project_ 1\faster-RCNN\mdimage\ data.png ]

  • 3. Running scripts for training and testing

    Return to the home directory and enter the following commands for training and testing:

    cd $FRCN_ROOT
    ./experiments/scripts/faster_rcnn_end2end.sh $DEVICE $DEVICE_ID VGG16 pascal_voc
    #Example of last statement
    ./experiments/scripts/faster_rcnn_end2end.sh gpu 0 VGG16 pascal_voc
    

    Possible problems and solutions in this step:

    Question 1: Authority

    bash: ./experiments/scripts/faster_rcnn_end2end.sh: Permission denied
    

    Solution: enter the following command at the terminal

    chmod 777 ./experiments/scripts/faster_rcnn_end2end.sh
    

    Problem 2: format of sh file

    /bin/bash^M: bad interpreter: No such file or directory
    

    Solution: change the sh file format from doc format to unix format. For details, refer to https://blog.csdn.net/xuforeverlove/article/details/85334726?utm_medium=distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.nonecase&depth_1-utm_source=distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.nonecase

    Problem 3: file index error

    If you use the official method, the program may not find it train.txt Errors are reported as follows:

    AssertionError: Path does not exist: /home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/data/VOCdevkit2007/VOC2007/ImageSets/Main/trainval.txt
    

    Solution: here are two solutions

    1. Reestablish the file link according to the official method, pay attention to the folder path and the linked file name

    2. According to the method I provided, change the VOCdevkit folder to VOCdevkit 2007 and put it on frcnn_ Under the root / data path.

    Problem 4: model file saving error

    Because. / experiences / scripts / foster_ rcnn_ end2 end.sh On file_ rcnn_ end2end_ faster_ rcnn_ end2end_ VGG16_ .2020-06-19_ The path of 14-46-49.txt 'is set incorrectly. The program cannot find the path and cannot create the file. The error is reported as follows:

    grep: experiments/logs/faster_rcnn_end2end_VGG16_.2020-06-19_14-46-49.txt: No such file or directory
    

    Solution: create logs folder under. / experiments folder or modify. / experiments / scripts / foster_ rcnn_ end2 end.sh For the definition of LOG variable in the file, change the path to the existing path.

    Problem 5: due to the change of the old and new versions, the model saving files become three (the old version is one) so as to test_net.py Wrong number of parameters accepted

    The error is reported as follows:

    usage: test_net.py [-h] [--device DEVICE] [--device_id DEVICE_ID]
                       [--def PROTOTXT] [--weights MODEL] [--cfg CFG_FILE]
                       [--wait WAIT] [--imdb IMDB_NAME] [--comp]
                       [--network NETWORK_NAME]
    test_net.py: error: argument --weights: expected one argument
    

    Solution: change the file ROOT/lib/fast_rcnn/train.py Trace function_ Net self.saver = tf.train.Saver(max_to_keep=100) to self.saver = tf.train.Saver(max_to_keep=100,write_version=saver_pb2.SaverDef.V1) and add from at the front of the file tensorflow.core.protobuf import saver_ PB2

    The normal operation results are as follows:

    VOC07 metric? Yes
    AP for aeroplane = -1.0000
    AP for bicycle = -1.0000
    AP for bird = -1.0000
    AP for boat = -1.0000
    AP for bottle = -1.0000
    AP for bus = 0.0075
    AP for car = 0.0288
    AP for cat = 0.0130
    AP for chair = 0.0036
    AP for cow = -1.0000
    AP for diningtable = -1.0000
    AP for dog = 0.0000
    AP for horse = 0.0048
    AP for motorbike = 0.0035
    AP for person = 0.0489
    AP for pottedplant = 0.0068
    AP for sheep = -1.0000
    AP for sofa = -1.0000
    AP for train = 0.0091
    AP for tvmonitor = -1.0000
    Mean AP = -0.4937
    ~~~~~~~~
    Results:
    -1.000
    -1.000
    -1.000
    -1.000
    -1.000
    0.008
    0.029
    0.013
    0.004
    -1.000
    -1.000
    0.000
    0.005
    0.003
    0.049
    0.007
    -1.000
    -1.000
    0.009
    -1.000
    -0.494
    ~~~~~~~~
    
    --------------------------------------------------------------
    Results computed with the **unofficial** Python eval code.
    Results should be very close to the official MATLAB eval code.
    Recompute with `./tools/reval.py --matlab ...` for your paper.
    -- Thanks, The Management
    --------------------------------------------------------------
    
    real	8m0.321s
    user	97m19.020s
    sys	1m43.796s
    

Train your own dataset

For detailed steps, see https://blog.csdn.net/zcy0xy/article/details/79614862

Problems encountered and Solutions

Question 1

Traceback (most recent call last):
  File "./tools/train_net.py", line 96, in <module>
    max_iters=args.max_iters)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 260, in train_net
    roidb = filter_roidb(roidb)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 251, in filter_roidb
    filtered_roidb = [entry for entry in roidb if is_valid(entry)]
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 240, in is_valid
    overlaps = entry['max_overlaps']
KeyError: 'max_overlaps'

terms of settlement:

Delete the folders ROOT/data/cache and root / data / vocdevkit 2007 / annotations_ cache

Question 2

Computing bounding-box regression targets...
Traceback (most recent call last):
  File "./tools/train_net.py", line 96, in <module>
    max_iters=args.max_iters)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 263, in train_net
    sw = SolverWrapper(sess, saver, network, imdb, roidb, output_dir, pretrained_model=pretrained_model)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 39, in __init__
    self.bbox_means, self.bbox_stds = rdl_roidb.add_bbox_regression_targets(roidb)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/roi_data_layer/roidb.py", line 48, in add_bbox_regression_targets
    assert len(roidb) > 0
AssertionError

terms of settlement:

There is a problem with the record format of the index file. According to PASCAL VOC 2007, view the difference in the record format of the index file, and modify generate according to the difference_ txt.py Document related code.

Question three

Traceback (most recent call last):
  File "./tools/train_net.py", line 83, in <module>
    roidb = get_training_roidb(imdb)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 205, in get_training_roidb
    imdb.append_flipped_images()
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/datasets/imdb.py", line 106, in append_flipped_images
    widths = self._get_widths()
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/datasets/imdb.py", line 102, in _get_widths
    for i in xrange(self.num_images)]
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/datasets/pascal_voc.py", line 64, in image_path_at
    return self.image_path_from_index(self._image_index[i])
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/datasets/pascal_voc.py", line 73, in image_path_from_index
    'Path does not exist: {}'.format(image_path)
AssertionError: Path does not exist: /home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/data/VOCdevkit2007/VOC2007/JPEGImages/b38.jpg

terms of settlement:

This is a problem caused by the unconventional case of the suffix name of the picture file. You can change the name of the picture file, change all the suffix names to lower case jpg, or change ROOT/lib/datasets/pascal_voc.py The specific changes are as follows:

take
self._image_ext = '.jpg'
//Change to:
self._image_ext = '.JPG'

Question 4

Traceback (most recent call last):
  File "./tools/train_net.py", line 83, in <module>
    roidb = get_training_roidb(imdb)
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/fast_rcnn/train.py", line 205, in get_training_roidb
    imdb.append_flipped_images()
  File "/home/qianslab/linbin/chaosheng/Faster_RCNN/Faster-RCNN_TF-python2.7/tools/../lib/datasets/imdb.py", line 108, in append_flipped_images
    boxes = self.roidb[i]['boxes'].copy()
IndexError: list index out of range

terms of settlement:

The ROOT/data/cache folder is not updated. You only need to delete the folder and update it again.

Tags: Python github network MATLAB

Posted on Fri, 26 Jun 2020 00:10:24 -0400 by silentmajority3