Knapsack problem is a kind of classic dynamic programming problem. It is very flexible and needs careful consideration and experience. This paper first summarizes several common types of knapsack problem, then gives the code template, and then looks at several related topics on LeetCode.

according to Wikipedia , Knapsack problem is a NP complete (NPC) problem of combinatorial optimization. The problem can be described as: given a group of items, each item has its own weight and price. Within the limited total weight, how can we choose to make the total price of the item the highest. NPC problem has no polynomial time complexity, but using dynamic programming, we can solve Knapsack problem with pseudo polynomial time complexity. Generally speaking, knapsack problems can be classified as follows:

1. 01 knapsack problem
2. Complete knapsack problem
3. Multiple knapsack problem

In addition, there are some other test methods, such as just full, finding the total number of schemes, finding all schemes, etc. Next, this paper discusses these problems respectively.

1. 01 Backpack

1.1 title

The most basic knapsack problem is 01 knapsack problem: there are N items in total. The weight of the I (I starts from 1) item is w[i] and the value is v[i]. When the total weight does not exceed the upper limit W of the backpack, what is the maximum value that can be loaded into the backpack?

1.2 analysis

If the violent exhaustive method is adopted, there are two situations of loading and not loading each item, so the total time complexity is O(2^N), which is unacceptable. Dynamic programming can reduce the complexity to O(NW). Our goal is the total value of the items in the schoolbag, and the variable is the weight limit of the items and schoolbag, so we can define the state dp:

dp[i][j]Indicates that before i The weight limit for loading articles is j The maximum value you can get from your backpack, 0<=i<=N, 0<=j<=W

Then we can initialize dp[0][0... W] to 0, which means that the maximum value of loading the first 0 items (i.e. no items) into the schoolbag is 0. When I > 0, dp[i][j] there are two situations:

1. Do not load the ith article, i.e. dp[i − 1][j];
2. Load the ith item (provided it can be loaded), i.e. dp[i − 1][j − w[i]] + v[i].

That is, the state transition equation is:

dp[i][j] = max(dp[i−1][j], dp[i−1][j−w[i]]+v[i]) // j >= w[i]

From the above state transition equation, it can be seen that the value of dp[i][j] is only related to dp[i-1][0,...,j-1], so we can use the common method of dynamic programming (rolling array) to optimize the space (i.e. remove the first dimension of dp). It should be noted that in order to prevent the dp[0,...,j-1] of the upper layer loop from being overwritten, j can only be inversely enumerated during the loop (there is no restriction before space optimization). The pseudo code is:

// 01 knapsack problem pseudo code (space optimization version)
dp[0,...,W] = 0
for i = 1,...,N
    for j = W,...,w[i] // Must reverse enumeration!!!
        dp[j] = max(dp[j], dp[j−w[i]]+v[i])

The time complexity is O(NW) and the space complexity is O(W). Since the value of W is a power of the number of bits of W, this time complexity is pseudo polynomial time.

The core idea of dynamic programming is to avoid repeated calculation, which is reflected incisively and vividly in 01 knapsack problem. The maximum value obtained by loading or not loading the ith item can be determined by the maximum value of the previous i-1 item, which is ignored by violence enumeration.

2. Complete Backpack

2.1 title

The difference between the unbounded knapsack problem and the 01 knapsack is that each item can have an unlimited number: there are N items in total, and each item has an unlimited number. The weight of the I (I starts from 1) item is w[i], and the value is v[i]. When the total weight does not exceed the upper limit W of the backpack, what is the maximum value that can be loaded into the backpack?

2.2 analysis I

Our goals and variables are no different from the 01 knapsack problem, so we can define almost the same state dp as the 01 knapsack problem:

dp[i][j]Indicates that before i The loading weight limit for each article is j The maximum value you can get from your backpack, 0<=i<=N, 0<=j<=W

The initial state is the same. We initialize dp[0][0... W] to 0, which means that the maximum value of loading the first 0 items (i.e. no items) into the schoolbag is 0. Then when I > 0, dp[i][j] there are two cases:

1. Do not load item I, i.e. dp[i − 1][j], the same as 01 backpack;
2. Loading item I is different from 01 backpack at this time, because there are infinite items (but note that the weight limit of schoolbag is limited), so it should not be transferred to dp[i − 1][j − w[i]] but to dp[i][j − w[i]], that is, you can continue to load item I after loading item I.

So the state transition equation is:

dp[i][j] = max(dp[i−1][j], dp[i][j−w[i]]+v[i]) // j >= w[i]

The only difference between this state transition equation and the 01 knapsack problem is that the second term of max is not dp[i-1] but dp[i].

Similar to the 01 knapsack problem, space optimization can also be carried out. The difference after optimization is that j here can only be enumerated in the forward direction and 01 knapsack can only be enumerated in the reverse direction, because the second term of max here is dp[i] and 01 knapsack is dp[i-1], that is, coverage is needed here and 01 knapsack needs to be avoided. Therefore, the pseudo code is as follows:

// Complete knapsack problem idea I pseudo code (space optimization version)
dp[0,...,W] = 0
for i = 1,...,N
    for j = w[i],...,W // Must enumerate forward!!!
        dp[j] = max(dp[j], dp[j−w[i]]+v[i])

2.3 analysis II it can be seen from the above pseudo code that the only difference between the spatial optimization solution of 01 knapsack problem and complete knapsack problem is that the former j can only be enumerated inversely, while the latter j can only be enumerated forward, which is determined by their state transition equations. The time complexity of this solution is O(NW) and the space complexity is O(W).

In addition to the idea of analyzing one, complete knapsack has a common idea, but it is more complex. Starting from the number of items I loaded, there are only two cases for 01 backpack, that is, take 0 and take 1. Here, take 0, 1 and 2... Until the weight limit is exceeded (k > J / w [i]), so the state transition equation is:

# k is the number of i-th article loaded, k < = J / w [i]
dp[i][j] = max{(dp[i-1][j − k*w[i]] + k*v[i]) for every k}

Similarly, space optimization can also be carried out. It should be noted that dp[i-1] in max is the same as 01 knapsack, so j must be enumerated inversely. After optimization, the pseudo code is:

// Complete knapsack problem idea II pseudo code (space optimization version)
dp[0,...,W] = 0
for i = 1,...,N
    for j = W,...,w[i] // Must reverse enumeration!!!
        for k = [0, 1,..., j/w[i]]
            dp[j] = max(dp[j], dp[j−k*w[i]]+k*v[i])

Compared with analysis 1, this method does not obtain dp[i][j] at O(1), so the total time complexity is larger than analysis 1, which is $O(NW \frac W {\bar{w}}) $level.

2.4 analysis III. conversion to 01 Backpack

01 knapsack problem is the most basic knapsack problem. We can consider transforming the complete knapsack problem into 01 knapsack problem: converting an item into several items that can only be loaded into 0 or 1 01 knapsack.

The simplest idea is that considering that the i-th item can be loaded with W/w[i] pieces at most, the i-th item can be transformed into W/w[i] pieces with constant cost and value, and then the 01 knapsack problem can be solved.

A more efficient conversion method is to adopt the binary idea: divide the i-th item into $w_i 2^k $, value $V_ Several items of I 2 ^ k $, of which K satisfies $w_i 2^k \le W $. This is because no matter how many items I are selected by the optimal strategy, it can always be expressed as the sum of several items just now (e.g. 13 = 1 + 4 + 8). This reduces the number of converted items to the logarithmic level. See Section 3.4 template for specific code.

3. Multiple backpacks

3.1 title

The difference between the bounded knapsack problem and the previous one is that each item is limited: there are N items in total, the quantity of the I (I starts from 1) item is n[i], the weight is w[i], and the value is v[i]. When the total weight does not exceed the upper limit W of the backpack, what is the maximum value that can be loaded into the backpack?

3.2 analysis I

The analysis at this time is similar to the analysis II of the complete backpack. It also starts from the number of items of type I: 0 items, 1 items,... n[i] items of type I (not exceeding the weight limit). So the equation of state is:

# k is the number of i-th article loaded, k < = min (n [i], J / w [i])
dp[i][j] = max{(dp[i-1][j − k*w[i]] + k*v[i]) for every k}

Similarly, space optimization can also be carried out, and j must also be inversely enumerated. After optimization, the pseudo code is

// Complete knapsack problem idea II pseudo code (space optimization version)
dp[0,...,W] = 0
for i = 1,...,N
    for j = W,...,w[i] // Must reverse enumeration!!!
        for k = [0, 1,..., min(n[i], j/w[i])]
            dp[j] = max(dp[j], dp[j−k*w[i]]+k*v[i])

The total time complexity is about $O(NW{\bar{n}}) = O(W \sum_i {n_i}) $level.

3.3 analysis II. Conversion to 01 Backpack

Using the similar ideas in Section 2.4, the multiple knapsack problem can be transformed into the 01 knapsack problem. Using the binary idea, the I item is divided into $O(logn_i) $items, and the original problem is transformed into the 01 knapsack problem with a complexity of $O(W \sum_i{logn_i}). It is a great improvement compared with analysis I. see Section 3.4 for the specific code.

3.4 code template

This section gives the problem-solving templates of these three knapsack problems according to the above explanation, which is convenient for problem-solving. Pay particular attention to how binary optimization is implemented.

/*
https://tangshusen.me/2019/11/24/knapsack-problem/
01 Knapsack, complete knapsack, multiple knapsack template (binary optimization) 
2020.01.04 by tangshusen.

Usage:
    Call the corresponding function for each item, such as multiple backpacks:
    for(int i = 0; i < N; i++) 
        multiple_pack_step(dp, w[i], v[i], num[i], W);

Parameters:
    dp   : The space optimized one-dimensional dp array, namely dp[i], represents the result of the schoolbag with the maximum load of I
    w    : The weight of this item
    v    : The value of this item
    n    : Number of this item
    max_w: Maximum load of schoolbag
*/
void zero_one_pack_step(vector<int>&dp, int w, int v, int max_w){
    for(int j = max_w; j >= w; j--) // Reverse enumeration!!!
        dp[j] = max(dp[j], dp[j - w] + v);
}

void complete_pack_step(vector<int>&dp, int w, int v, int max_w){
    for(int j = w; j <= max_w; j++) // Forward enumeration!!!
        dp[j] = max(dp[j], dp[j - w] + v);

    // Method 2: convert to 01 knapsack, binary optimization
    // int n = max_w / w, k = 1;
    // while(n > 0){
    //     zero_one_pack_step(dp, w*k, v*k, max_w);
    //     n -= k;
    //     k = k*2 > n ? n : k*2;
    // }
}

void multiple_pack_step(vector<int>&dp, int w, int v, int n, int max_w){
   if(n >= max_w / w) complete_pack_step(dp, w, v, max_w);
   else{ // Convert to 01 knapsack, binary optimization
       int k = 1;
       while(n > 0){
           zero_one_pack_step(dp, w*k, v*k, max_w);
           n -= k;
           k = k*2 > n ? n : k*2;
       }
   }
}

4. Other circumstances

In addition to the above three basic knapsack problems, there are other variants, as shown in the figure below( picture source).

 

This section lists several common.

4.1 just full

Sometimes there is a limitation to the knapsack problem, that is, the knapsack must be filled exactly. At this time, the basic idea is not different, but it is different during initialization.

If there is no limit of just filling the backpack, we can initialize all dp to 0. Because the backpack of any capacity has a legal solution "nothing", the value of this solution is 0, so the initial state values are all 0. If there is a limit of just filling, only dp[0,..., N][0] The initial value is 0, and other dp values are initialized to - inf, because at this time, only backpacks with capacity of 0 can be "just filled" without loading anything. Backpacks with other capacity have no legal solution initially and should be initialized to - inf.

4.2 total number of schemes

In addition to finding the maximum value that can be obtained after giving the value of each item, another kind of problem is to ask the total number of schemes that fill the backpack or pack the backpack to a specified capacity. For this kind of problem, it is necessary to change max in the state transition equation to sum, and the general idea is unchanged. For example, if each item is completely an item in the backpack, the transition equation is:

dp[i][j] = sum(dp[i−1][j], dp[i][j−w[i]]) // j >= w[i]

4.3 two dimensional Backpack

The knapsack capacity discussed above is a quantity: weight. The two-dimensional knapsack problem means that each knapsack has two constraints (such as weight and volume constraints), which must be met when selecting items. The solution of this kind of problem is different from the one-dimensional knapsack problem, that is, the dp array needs to be more one-dimensional, and others are exactly the same as the one-dimensional knapsack, such as section 5.4.

4.4 finding the optimal scheme

Generally speaking, the knapsack problem requires an optimal value. If the scheme that requires to output this optimal value can refer to the method of outputting the scheme of the general dynamic programming problem: record which strategy deduces the optimal value of each state, so that the previous state can be found according to this strategy, and then push forward from the previous state.

Taking 01 knapsack as an example, we can use an array G[i][j] to record the scheme   G[i][j] = 0 means that the previous term in max (i.e. DP [I − 1] [J]) is used when calculating the value of DP [i] [J], G[i][j] = 1   Indicates that the latter term of the equation is adopted. That is, it indicates two strategies respectively: not loading the ith item and loading the ith item. In fact, we can also directly deduce the solution from the DP [i] [J]: if   dp[i][j] = dp[i−1][j]   It indicates that the i-th item is not selected, otherwise it indicates that it is selected.

5. LeetCode related topics

This section discusses the knapsack problem above LeetCode.

5.1 Partition Equal Subset Sum

416. Partition Equal Subset Sum

The question is to give a non empty array containing only positive integers. Ask whether the array can be divided into two subsets so that the sum of the elements of the two subsets is equal.

Since the sum of all elements is known, the sum of the two subsets should be sum/2 (so the premise is that sum cannot be an odd number), that is, the problem is to select some elements from this array to make the sum of these elements sum up to sum/2. If we regard the value of all elements as the weight of items, and the value of each item is 1, this is a 01 knapsack problem that is just full.

We define the space optimized state array dp. Because it is just full, we should initialize dp[0] to 0 and all others to INT_MIN, and then update dp according to the pseudo code similar to section 1.2:

int capacity = sum / 2;
vector<int>dp(capacity + 1, INT_MIN);
dp[0] = 0;
for(int i = 1; i <= n; i++)
    for(int j = capacity; j >= nums[i-1]; j--)
        dp[j] = max(dp[j], 1 + dp[j - nums[i-1]]);

After the update, if dp[sum/2] is greater than 0, it indicates that the meaning of the question is met.

Since the final question of this question is whether it can be divided, each element of dp can be defined as bool type, and then dp[0] is initialized to true, others are initialized to false, and the transfer equation should be operated by or rather than max. the complete code is as follows:

bool canPartition(vector<int>& nums) {
    int sum = 0, n = nums.size();
    for(int &num: nums) sum += num;
    if(sum % 2) return false;
    
    int capacity = sum / 2;
    vector<bool>dp(capacity + 1, false);
    dp[0] = true;
    for(int i = 1; i <= n; i++)
        for(int j = capacity; j >= nums[i-1]; j--)
            dp[j] = dp[j] || dp[j - nums[i-1]];
        
    return dp[capacity];
}

In addition, there is a more ingenious and faster solution to this problem. The basic idea is to use a bisets to record the sum of all possible subsets. See My Github.

5.2 Coin Change

322. Coin Change

Given a value amount and some denominations, assuming that the number of coins of each denomination is infinite, we can use at least a few coins to form a given value.

If we regard the face value as an item and the face value as the weight of the item, and the value of each item is 1, this problem is a complete knapsack problem that is just full. However, it is not the maximum number of items loaded here, but the minimum. We only need to change max in the state transition equation in Section 2.2 to min. since it is just full, all other items except dp[0] should be initialized to INT_MAX. The complete code is as follows:

int coinChange(vector<int>& coins, int amount) {
    vector<int>dp(amount + 1, INT_MAX);
    dp[0] = 0;
    
    for(int i = 1; i <= coins.size(); i++)
        for(int j = coins[i-1]; j <= amount; j++){
            // The downlink code will be in 1 + int_ Overflow on Max
            // dp[j] = min(dp[j], 1 + dp[j - coins[i-1]]); 
            if(dp[j] - 1 > dp[j - coins[i-1]])
                dp[j] = 1 + dp[j - coins[i-1]];   
        }
    return dp[amount] == INT_MAX ? -1 : dp[amount];   
}

Note that the above 1 + dp[j - coins[i-1]] will have the risk of overflow, so we wrote it differently.

In addition, this question can also search all possible results, and then maintain a global result res, but the direct search will timeout, so it needs careful pruning, and 99% can be defeated after pruning. See details My Github.

5.3 Target Sum

494. Target Sum

This question gives us an array (non negative element) and a target value. It is required to add a positive sign or negative sign before each number in the array. The expression result is equal to the target value S. how many cases are there.

Assuming that the sum of all elements is sum, the sum of all elements with positive signs is A, and the sum of all elements with negative signs is B, sum = A + B   And   S = A - B, solve the equation to get A = (sum + S)/2. That is, the topic is transformed into: select some elements from the array to make the sum exactly (sum + S) / 2. It can be seen that this is A 01 knapsack problem that is just full. It requires all the schemes to change the max in the state transition equation in section 1.2 to sum. It should be noted that although it is just full here, the initial value of dp should not be inf, because it is not the total value but the number of schemes, and all should be initialized to 0 (except that dp[0] is initialized to 1). So the code is as follows:

int findTargetSumWays(vector<int>& nums, int S) {
    int sum = 0;
    // for(int &num: nums) sum += num;
    sum = accumulate(nums.begin(), nums.end(), 0);
    if(S > sum || sum < -S) return 0; // Definitely not
    if((S + sum) & 1) return 0; // Odd number
    int target = (S + sum) >> 1;
    
    vector<int>dp(target + 1, 0);
    
    dp[0] = 1;
    for(int i = 1; i <= nums.size(); i++)
        for(int j = target; j >= nums[i-1]; j--)
            dp[j] = dp[j] + dp[j - nums[i-1]];
    
    return dp[target];
}

5.4 Ones and Zeros

The title gives an array containing only 0 and 1 strings. The task is to select as many strings as possible from the array so that the number of 0 and 1 contained in these strings does not exceed m and n respectively.

We regard each string as an item, and the number of 0 and 1 in the string as two "weights", so it becomes a two-dimensional 01 knapsack problem. The two weight limits of the schoolbag are m and n respectively, which requires the maximum number of items that the schoolbag can hold (also equivalent to the maximum value, set the value of each item as 1).

We can put two "weights" of each string in advance   w0 and w1 are calculated and stored in an array, but note that these two values only need to be used once, so we only need to calculate w0 and w1 when we use them, so we don't need to store additional arrays. The complete code is as follows:

int findMaxForm(vector<string>& strs, int m, int n) {
    int num = strs.size();
    int w0, w1;
    
    vector<vector<int>>dp(m+1, vector<int>(n+1, 0));
    
    for(int i = 1; i <= num; i++){
        w0 = 0; w1 = 0;
        // Calculate the two weights of the i-1 string
        for(char &c: strs[i - 1]){
            if(c == '0') w0 += 1;
            else w1 += 1;
        }
        
        // 01 knapsack, reverse iteration update dp
        for(int j = m; j >= w0; j--)
            for(int k = n; k >= w1; k--)
                dp[j][k] = max(dp[j][k], 1+dp[j-w0][k-w1]);
    }
    
    return dp[m][n];
}

6. Summary
This paper discusses several kinds of knapsack problems and LeetCode related problems. Among them, 01 knapsack problem and complete knapsack problem are the most frequently tested. In addition, we need to pay attention to some other variants, such as just full, two-dimensional knapsack, finding the total number of schemes and so on. In addition to these knapsack problems discussed in this paper, there are some other variants, but as long as we deeply understand the ideas and state transition equations of knapsack problems listed in this paper, it should not be difficult to come up with algorithms when encountering other deformation problems. If you want to understand the knapsack problem in more detail, it is recommended to read the classic knapsack problem nine.