I'm trying to derive a measure of Tumours heterogeneity in scRNA-seq data. For a given individual's scRNA-seq gene expression matrix, I could like to calculate the Pearson correlations with and between clusters (compare the average cell-to-cell correlation within clusters and the average cell-to-cell correlation between clusters).
The idea is that if an individual had substantial transcriptional heterogeneity, the intercluster correlation would be negative and within-cluster correlation would be positive. If the individual's Tumours had a uniform transcriptional state, they would have a near-zero intercluster correlation. I am using the cor() function from the R 'stats' package and using the log-normalized gene expression matrix as input:
c2.dat <- data.frame(c2#assays$RNA#data) #gene expression matrix for a subject's cells in "cluster 2"
c2.cor <- cor(c2.dat, method = "pearson") #correlation analysis on log-normalized gene expression matrix
I am stuck though once I have a correlation matrix. How do I calculate the average cell-to-cell correlation within this cluster?
Thank you :)
Predicting the probability of class assignment for each chosen sample from the Train_features:
probs = classifier.predict_proba(Train_features)`
Choosing the class for which the AUC has to be determined.
preds = probs[:,1]
Calculating false positive rate, true positive rate and the possible thresholds that can clearly separate TP and TN.
fpr, tpr, threshold = metrics.roc_curve(Train_labels, preds)
roc_auc = metrics.auc(fpr, tpr)
print(max(threshold))
Output : 1.97834
The previous answer did not really address your question of why the threshold is > 1, and in fact is misleading when it says the threshold does not have any interpretation.
The range of threshold should technically be [0,1] because it is the probability threshold. But scikit learn adds +1 to the last number in the threshold array to cover the full range [0, 1]. So if in your example the max(threshold) = 1.97834, the very next number in the threshold array should be 0.97834.
See this sklearn github issue thread for an explanation. It's a little funny because somebody thought this is a bug, but it's just how the creators of sklearn decided to define threshold.
Finally, because it is a probability threshold, it does have a very useful interpretation. The optimal cutoff is the threshold at which sensitivity + specificity are maximum. In sklearn learn this can be computed like so
fpr_p, tpr_p, thresh = roc_curve(true_labels, pred)
# maximize sensitivity + specificity, i.e. tpr + (1-fpr) or just tpr-fpr
th_optimal = thresh[np.argmax(tpr_p - fpr_p)]
The threshold value does not have any kind of interpretation, what really matters is the shape of the ROC curve. Your classifier performs well if there are thresholds (no matter their values) such that the generated ROC curve lies above the linear function (better than random guessing); your classifier has a perfect result (this happens rarely in practice) if for any threshold the ROC curve is only one point at (0,1); your classifier has the worst result if for any threshold the ROC curve is only one point at (1,0). A good indicator of the performance of your classifier is the integral of the ROC curve, this indicator is known as AUC and is limited between 0 and 1, 0 for the worst performance and 1 for perfect performance.
In the CS231n course of Stanford (http://cs231n.github.io/linear-classify/#loss), we train a linear classifier on images from CIFAR-10, and when plotting the weight matrix for each class, we get an image that resembles the class predicted. How is that a consequence of the training process? What is the mathematical reason behind it ?
Linear classifier: f(x, W, b) = W*x + b
input x is a flattened image
each row of W can be reconstructed to become an image
I am going to solve the following nonlinear DE:
Code#1:
tspan1 =t0:0.05:TT;
[t1,y1] = ode45(#(t1,T) ((1-alpha)*Q-sigm*(T.^4))/R, tspan1, t0);
h1=(TT-t0)/(size(y1,1)-1);
Tspan1=t0:h1:TT;
figure(55);plot(Tspan1,y1,'b');
Code#2:
tspan=[t0 TT];
[t,y] = ode45(#(t,T) ((1-alpha)*Q-sigm*(T.^4))/R, tspan, t0);
h=(TT-t0)/(size(y,1)-1);
Tspan=t0:h:TT;
figure(5);plot(Tspan,y,'b');
wherein:
R=2.912;
Q = 342;
alpha=0.3;
sigm=5.67*(10^(-8));
TT=20;
t0=0;
why the results are different?
The second result is not equally spaced. It in some way a minimal set of points that represents the solution curve. So if the curve is rather linear, there will be only few points, while at regions of high curvature you get a dense sampling. You can and should use the returned time array, as that contains the times that the solution points are for,
figure(55);plot(t1,y1,'b');
figure(5);plot(t,y,'b');
I am implementing various Backpropagation algorithms for the same dataset and trying to compare the performance. I got a help from the following tutorial for the same.
https://nl.mathworks.com/help/nnet/ug/choose-a-multilayer-neural-network-training-function.html
I tried to plot:
mean square error versus execution time for each algorithm
time required to converge versus the mean square error convergence goal for each algorithm
Have used the following code, to create my neural network and willing to know how can i implement the above two plots.
%Data
x=0:0.2:6*pi; y=sin(x);
p=con2seq(x); t=con2seq(y);
% Networks
net1=feedforwardnet(20,'trainlm');
net2=feedforwardnet(20,'traingd');
net2.iw{1,1}=net1.iw{1,1}; %set the same weights and biases for the networks
net2.lw{2,1}=net1.lw{2,1};
net2.b{1}=net1.b{1};
net2.b{2}=net1.b{2};
%training and simulation
net1.trainParam.epochs=1; % set the number of epochs for the training
net2.trainParam.epochs=1;
net1=train(net1,p,t); % train the networks
net2=train(net2,p,t);
a11=sim(net1,p); a21=sim(net2,p); % simulate the networks with the input vector p
net1.trainParam.epochs=14;
net2.trainParam.epochs=14;
net1=train(net1,p,t);
net2=train(net2,p,t);
a12=sim(net1,p); a22=sim(net2,p);
net1.trainParam.epochs=985;
net2.trainParam.epochs=985;
net1=train(net1,p,t);
net2=train(net2,p,t);
a13=sim(net1,p); a23=sim(net2,p);
%plots
figure
subplot(3,3,1);
plot(x,y,'bx',x,cell2mat(a11),'r',x,cell2mat(a21),'g'); % plot the sine function and the output of the networks
title('1 epoch');
legend('target','trainlm','traingd');
subplot(3,3,2);
postregm(cell2mat(a11),y); % perform a linear regression analysis and plot the result
subplot(3,3,3);
postregm(cell2mat(a21),y);
%
subplot(3,3,4);
plot(x,y,'bx',x,cell2mat(a12),'r',x,cell2mat(a22),'g');
title('15 epochs');
legend('target','trainlm','traingd');
subplot(3,3,5);
postregm(cell2mat(a12),y);
subplot(3,3,6);
postregm(cell2mat(a22),y);
%
subplot(3,3,7);
plot(x,y,'bx',x,cell2mat(a13),'r',x,cell2mat(a23),'g');
title('1000 epochs');
legend('target','trainlm','traingd');
subplot(3,3,8);
postregm(cell2mat(a13),y);
subplot(3,3,9);
postregm(cell2mat(a23),y);
Note that MSE is used by default when no measure of error is specified.
When training you can do something like:
[net tr] = train(net, x, t);
Then plot tr.perf or tr.tperf or tr.vperf