Question

In: Computer Science

Consider a simple case where we have trained a NN for recognising patterns X1 to Xn...

  1. Consider a simple case where we have trained a NN for recognising patterns X1 to Xn belonging to some classes, say A and B. Suppose we have a new training data Xn+1 that belongs to class C. We want this also to be trained. Will the previous training be impacted? How would it work?

Solutions

Expert Solution

We have a model with those 2 classes and lets say it is the base_model. We have weights from the trained model. Considering the situation, we need to define a new model because the classification will be from 3 classes(A,B,C) rather than 2. So we need a new NN, old one wont work. In the new NN we will use the weights from the base-model. A NN require some weights for training. One usually use random weights in the beginning that are then altered as the the model is trained. So those trained weights from base-model will be used initially in new model and then the new model will be trained again for the new data of 3 classes. Previous training wont be impacted, the model will further increase its knowledge base. this is the basic essence of transfer learning.

Also, Freeze all the weights in original layers by allowing "new" model to optimize only new weights. That will give you exactly same predictive power for original 2 classes and might give OK performance for new ones.

OR

Train whole network at once (by propagating error of new class), which might be working for new class(es), but you will end up with ineffective original solution for 2 classes (since weights will be changed for the lower classes and final layer won't be updated to match those changes).


Related Solutions

Consider a simple case where we have trained a NN for recognising patterns X1 to Xn...
Consider a simple case where we have trained a NN for recognising patterns X1 to Xn belonging to some classes, say A and B. Suppose we have a new training data Xn+1 that belongs to class C. We want this also to be trained. Will the previous training be impacted? How would it work?
We have a random sample of observations on X: x1, x2, x3, x4,…,xn. Consider the following...
We have a random sample of observations on X: x1, x2, x3, x4,…,xn. Consider the following estimator of the population mean: x* = x1/2 + x2/4 + x3/4. This estimator uses only the first three observations. a) Prove that x* is an unbiased estimator. b) Derive the variance of x* c) Is x* an efficient estimator? A consistent estimator? Explain.
Suppose we have a random sample of n observations {x1, x2, x3,…xn}. Consider the following estimator...
Suppose we have a random sample of n observations {x1, x2, x3,…xn}. Consider the following estimator of µx, the population mean. Z = 12x1 + 14x2 + 18x3 +…+ 12n-1xn−1 + 12nxn Verify that for a finite sample size, Z is a biased estimator. Recall that Bias(Z) = E(Z) − µx. Write down a formula for Bias(Z) as a function of n and µx. Is Z asymptotically unbiased? Explain. Use the fact that for 0 < r < 1, limn→∞i=1nri...
Consider a random sample (X1, Y1), (X2, Y2), . . . , (Xn, Yn) where Y...
Consider a random sample (X1, Y1), (X2, Y2), . . . , (Xn, Yn) where Y | X = x is modeled by Y=β0+β1x+ε, ε∼N(0,σ^2), where β0,β1and σ^2 are unknown. Let β1 denote the mle of β1. Derive V(βhat1).
Suppose X1,···, Xn ∼ Exp(λ) are independent. What is the distribution of X1/S where S =...
Suppose X1,···, Xn ∼ Exp(λ) are independent. What is the distribution of X1/S where S = X1+X2+···+Xn? Please show me how to do this without using the property of chi-squared dist.
Suppose X1,···, Xn ∼ Exp(λ) are independent. What is the distribution of X1/S where S =...
Suppose X1,···, Xn ∼ Exp(λ) are independent. What is the distribution of X1/S where S = X1+X2+···+Xn? Please show me how to do this without using the property of chi-squared dist.
Let X1, . . . , Xn ∼ iid Exp(θ) and consider the test for H0...
Let X1, . . . , Xn ∼ iid Exp(θ) and consider the test for H0 : θ ≥ θ0 vs H1 : θ < θ0. (a) Find the size-α LRT. Express the rejection region in the form of R = {X > c ¯ } where c will depend on a value from the χ 2 2n distribution. (b) Find the appropriate value of c. (c) Find the formula for the P-value of this test. (d) Compare this test...
Consider the independent observations x1, x2, . . . , xn from the gamma distribution with...
Consider the independent observations x1, x2, . . . , xn from the gamma distribution with pdf f(x) = (1/ Γ(α)β^α)x^(α−1)e ^(−x/β), x > 0 and 0 otherwise. a. Write out the likelihood function b. Write out a set of equations that give the maximum likelihood estimators of α and β. c. Assuming α is known, find the likelihood estimator Bˆ of β. d. Find the expected value and variance of Bˆ
Consider n numbers x1, x2, . . . , xn laid out on a circle and...
Consider n numbers x1, x2, . . . , xn laid out on a circle and some value α. Consider the requirement that every number equals α times the sum of its two neighbors. For example, if α were zero, this would force all the numbers to be zero. (a) Show that, no matter what α is, the system has a solution. (b) Show that if α = 1 2 , then the system has a nontrivial solution. (c) Show...
Suppose X1, ..., Xn are i.i.d. from an exponential distribution with mean θ. If we are...
Suppose X1, ..., Xn are i.i.d. from an exponential distribution with mean θ. If we are testing H0 : θ = θ0 vs Ha : θ > θ0. Suppose we reject H0 when ( X¯n/ θ0) > 1 + (1.645/ √n) (a) (10 points) Calculate the power function G(ζ). You may leave your answer in terms of the standard normal cdf Φ(x). (b) (5 points) Is this test consistent?
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT