I have read that citation networks are associated with the complex adaptive networks as they exhibit following characteristics: self-organization, emergence, no-linearity, order/chaos dynamic, and are adaptive to its environment. I want to know exactly which feature of citations exhibits each of these (just a brief overview).
I also know that citation networks follow power-law and small world properties of complex networks. They also have high clustering coefficient.
In citation networks, documents interact in a non-linear fashion, leading to emergent patterns which are difficult to understand simply by considering the individual document. In-degree of the citations follow a power law; a few documents are highly-cited and the majority of documents have low or no citations. There is no centralized control in citation networks. While the behaviour of citations may change, evolve over time, or adapt to its changing environment, the citations and bibliographic coupling are essentially a Complex Adaptive System (CAS).
As I understand it depends on what feature you are looking for in the network. In some cases when you try to see the flow or percolation in citation networks you might find non-linearity emerging. Adaptiveness in the network has to deal with optimization of some network feature.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I saw that, while using the conv2d function of Theano, the filters were flipped both vertically and horizontally. Why is that? And does it matter in the case of a Convolutional Neural Network?
Because this is how convolution is defined mathematically. Without the flipping of filter, the operation is called cross-correlation. The advantage of convolution is that it has nicer mathematical properties.
However in the context of Convolutional Neural Network it doesn't matter whether you use convolution or cross-correlation, they are equivalent. This is because the weights of filters are learned during the training, i.e. they are updated to minimize a cost function. In a CNN that uses the cross-correlation operation, the learned filters will be equal to the flipped learned filters of a CNN that uses the convolution operation (assuming exactly the same training conditions for both, i.e. same initialization, inputs, number of epochs etc.). So the outputs of such two CNNs will be the same for the same inputs.
Cross-correlation operation is slightly more intuitive and simpler to implement (because no flipping is performed) and that's probably the reason why other frameworks like Tensorflow and Pytorch use it instead of the actual convolution (they still call it convolution though, probably due to historical reasons or to be consistent in terminology with other frameworks that use the actual convolution).
Just to add in to the above post, even though we say we are using "convolution" operation in CNNs, we actually use cross correlation. But both convolution and correlation produces different outcomes, and I did this exercise to actually see the difference in outcomes.
When I reserached more on this topic, I found that the very initial inception of CNNs is considered to be originated from the paper Neocognitron, which is known to use convolutional operation while using the kernel operation, but later implementations of CNNs and most Deep Learning libraries are known to use correlation filtering instead of convolutional filtering, and we still kept using the name of Convolutional Neural Networks as mostly the algorithm complexities and performance remained almost same.
If you want a detailed article and intuition on how both differ, please take a look at this: https://towardsdatascience.com/does-cnn-represent-convolutional-neural-networks-or-correlational-neural-networks-76c1625c14bd
Let me know if this helps. :)
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
Assume an ANN has trained by 1GB size training data using a long time.
Do we need train the data again when just a few rows in the training data changed?
or is the design of ANN error?
In general, the answer would be yes. Here's why...
First of all, 1GB of training data is a relative measure, since it gives no indication of the number of training samples. Perhaps each training sample is 1MB in size (maybe an image), leaving you with only about 1,000 samples which may not be enough.
Secondly, it is important to know the architecture of the neural network in order to address the question of retraining fully. If the components in your training set that was updated correspond to nodes that may be heavily influenced in terms of the usage, then a retrain is most certainly in order. Of course, the converse is not necessarily true, as it may not be immediately apparent how the interconnectedness of neural network may be influenced by a change in the input.
Thirdly, a neural network is meant to represent a type of complex pattern-matcher, trained to recognize some input relationship, and produce a possible output relationship. From this naive point of view, a change in input should surely correspond to a change in output. As such, a change in the training data may very well correspond to a change in the expected output data. And, even if it doesn't, the input pattern has changed, which may imply that the output pattern also changed.
Let's consider the following example. Assume your neural network is trained to match paintings with artists and it's been success in making the following match to Leonardo da Vinci:
Now it may be trained well enough to also assert that the following images are "painted" by the same artist:
This may be because you trained your neural network on your favourite past-time of balloon an Lego figurines. However, now a number of your input samples change, specifically those associated with the Mona Lisa. Instead they resemble your new favourite past-time... freehand mouse-drawing:
Despite what you say, in general the artistry of the above image doesn't really match that of the earlier ones. As such, your pattern-matcher may not appropriately recognize this as a piece of art made by Leonardo da Vinci. So it's fair to say that retraining it on images of this sort should be in order.
You probably have a number of options:
Test how effective it is to retrain your neural network given the change in training data. That would allow you to answer the question yourself, and give some insight into the architecture of your neural network.
Retrain your neural network on just the changes, which could be considered new training data. The value of this might depend heavily on your architecture, the complexity of the samples as well as the training quantity (not size).
Perform a complete retrain and test the efficacy.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I want to measure semantic similarity between two phrases/sentences. Is there any framework that I can use directly and reliably?
I have already checked out this question, but its pretty old and I couldn't find real helpful answer there. There was one link, but I found this unreliable.
e.g.:
I have a phrase: felt crushed
I have several choices: force inwards,pulverized, destroyed emotionally, reshaping etc.
I want to find the term/phrase with highest similarity to the first one.
The answer here is: destroyed emotionally.
The bigger picture is: I want to identify which frame from FrameNet matches to the given verb as per its usage in a sentence.
Update : I found this library very useful for measuring similarity between two words. Also the ConceptNet similarity mechanism is very good.
and this library for measuring semantic similarity between sentences
If anyone has any insights please share.
This is a very complicated problem.
The main technique that I can think of (before going into more complicated NLP processes) would be to apply cosine (or any other metric) similarity to each pair of phrases. Obviously this solution would be very inefficient at the moment due to the non-matching problem: The sentences might refer to the same concept with different words.
To solve this issue, you should transform the initial representation of each phrase with a more "conceptual" meaning. One option would be to extend each word with its synonyms (i.e. using WordNet, another option is to apply metrics such as distributional semantics DS (http://liawww.epfl.ch/Publications/Archive/Besanconetal2001.pdf) that extend the representation of each term with the more likely words to appear with it.
Example:
A representation of a document: {"car","race"} would be transform to {"car","automobile","race"} with synonyms. While, with DS it would be something like: {"car","wheel","road","pilot", ...}
Obviously this transformation won't be binary. Each term will have some associated weights.
I hope this helps.
Maybe the cortical.io API could help with your problem. The approach here is that every word is converted into a semantic fingerprint that characterizes the meaning of it with 16K semantic features. Phrases, sentences or longer texts are converted into fingerprints by ORing the word fingerprints together. After this conversion into a (numeric) binary vector representation semantic distance can easily be computed using distance measures like Euclidian Distance or cosine-similarity.
All necessary conversion- and comparison-functions are provided by the api.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 days ago.
Improve this question
Some e-commerce platforms have the suggestion feature where they tell you once you have an item in the basket that "you might like this product as well". Some, like Amazon, rely on the preexisting data on customer behaviour and their feature is called "Customers Who Bought This Item Also Bought" but some seem to suggest by other means.
What are these "other means"? What kind of algorithms do they use in webstores for this capability?
They use data mining, and this particular algorithm you're asking about is called the "nearest neighbor" algorithm.
Here's a link to an article I wrote on the algorithm (as well as many others).
http://www.ibm.com/developerworks/opensource/library/os-weka3/index.html
The process is called Business Intelligence, data will be stored in a data warehouse and the business intelligence process can be used using a product such as SSAS. The process will involve grouping the volumes of data (Who bought what and when) into data cubes. Analysis is performed on these cubes and used to compare your purchases with others who bought the same product, it will then recommend their purchases (Other customers who bought this, also bought this item....Item X). Other various AI algorithms are used to compare patterns across other customer trends such as how they shop, where they click etc. All this data is accumulated and then added to the data cube for analysis.
The data mining algorithms are outlined below, you could look for the Decision Tree Modelling algorithm which is how BI determines trends and patterns (In this case, Recommendations):
http://msdn.microsoft.com/en-us/library/ms175595.aspx
Majority of suggestions on e-commerce pages are created using some sort of a recommender system (http://en.wikipedia.org/wiki/Recommender_system). There are tools like Mahout (http://mahout.apache.org/) which already have implementation of most common approaches.
the best book about this kind of algorithms is: Programming Collective Intelligence
As some of the earlier folks answered, this is called recommendation engine. It is also referred to as Collaborative Filtering technique. There are few tools which does this, Mahout is one of them. Refer to the blog that I have written which talks about a use case where we use Mahout and Hadoop to build a recommendation engine. As a precursor to this, I have also written a Component architecture of how each of these fit together for a data mining problem.
Mahout will work in standalone mode and also with Hadoop. The decision to use either one really boils down to the size of the historical data that needs to be mined. If the data size is of the order of Terabytes and Petabytes, you typically use Mahout with Hadoop. Weka is another similar open source projects. All these come under a category called machine learning frameworks. I hope it help.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I have been learning alot about using graphs for machine learning by watching Christopher Bishops videos( http://videolectures.net/mlss04_bishop_gmvm/ ). I find it very interesting and watched a few others in the same categories(machine learning/graph) but was wondering if anyone had any recommendations for ways of learning more?
My problem is, although the videos gave a great high level understanding, I don't have much practical skills in it yet. I've read Bishops book on machine learning/patterns as well as Norvig's AI book but both don't seem to touch upon specific using graphs much. With the emergence of search engines and social networking, I would think machine learning on graphs would be popular.
If possible, can anyone suggestion an a resource to learn from? (I'm new to this field and development is a hobby for me, so I'm sorry in advance if there's a super obvious resource to learn from..I tried google and university sites).
Thanks in advance!
First, i would strongly recommend the book Social Network Analysis for Startups by Maksim Tsvetovat and Alexander Kouznetsov. A book like this is a godsend for programmers who need to quickly acquire a basic fluency in a specific discipline (in this case, graph theory) so that they can begin writing code to solve problems in this domain. Both authors are academically trained graph theoreticians but the intended audience of their book is programmers. Nearly all of the numerous examples presented in the book are in python using the networkx library.
Second, for the projects you have in mind, two kinds of libraries are very helpful if not indispensible:
graph analysis: e.g., the excellent networkx (python), or igraph
(python, R, et. al.) are two that i can recommend highly; and
graph rendering: the excellent graphViz, which can be used
stand-alone from the command line but more likely you will want to
use it as a library; there are graphViz bindings in all major
languages (e.g., for python there are at least three i know of,
though pygraphviz is my preference; for R there is rgraphviz which is
part of the bioconductor package suite). Rgraphviz has excellent documentation (see in particular the Vignette included with the Package).
It is very easy to install and begin experimenting with these libraries and in particular using them
to learn the essential graph theoretic lexicon and units of analysis
(e.g., degree sequence distribution, nodes traversal, graph
operators);
to distinguish critical nodes in a graph (e.g., degree centrality,
eigenvector centrality, assortivity); and
to identify prototype graph substructures (e.g., bipartite structure,
triangles, cycles, cliques, clusters, communities, and cores).
The value of using a graph-analysis library to quickly understand these essential elements of graph theory is that for the most part there is a 1:1 mapping between the concepts i just mentioned and functions in the (networkx or igraph) library.
So e.g., you can quickly generate two random graphs of equal size (node number), render and then view them, then easily calculate for instance the average degree sequence or betweenness centrality for both and observer first-hand how changes in the value of those parameters affects the structure of a graph.
W/r/t the combination of ML and Graph Theoretic techniques, here's my limited personal experience. I use ML in my day-to-day work and graph theory less often, but rarely together. This is just an empirical observation limited to my personal experience, so the fact that i haven't found a problem in which it has seemed natural to combine techniques in these two domains. Most often graph theoretic analysis is useful in ML's blind spot, which is the availability of a substantial amount of labeled training data--supervised ML techniques depend heavily on this.
One class of problems to illustrate this point is online fraud detection/prediction. It's almost never possible to gather data (e.g., sets of online transactions attributed to a particular user) that you can with reasonable certainty separate and label as "fraudulent account." If they were particularly clever and effective then you will mislabel as "legitimate" and for those accounts for which fraud was suspected, quite often the first-level diagnostics (e.g., additional id verification or an increased waiting period to cash-out) are often enough to cause them to cease further activity (which would allow for a definite classification). Finally, even if you somehow manage to gather a reasonably noise-free data set for training your ML algorithm, it will certainly be seriously unbalanced (i.e., much more "legitimate" than "fraud" data points); this problem can be managed with statistics pre-processing (resampling) and by algorithm tuning (weighting) but it's still a problem that will likely degrade the quality of your results.
So while i have never been able to successfully use ML techniques for these types of problems, in at least two instances, i have used graph theory with some success--in the most recent instance, by applying a model adapted from the project by a group at Carnegie Mellon initially directed to detection of online auction fraud on ebay.
MacArthur Genius Grant recipient and Stanford Professor Daphne Koller co-authored a definitive textbook on Bayesian networks entitled Probabalistic Graphical Models, which contains a rigorous introduction to graph theory as applied to AI. It may not exactly match what you're looking for, but in its field it is very highly regarded.
You can attend free online classes at Stanford for machine learning and artificial intelligence:
https://www.ai-class.com/
http://www.ml-class.org/
The classes are not simply focused on graph theory, but include a broader introduction in the field and they will give you a good idea of how and when you should apply which algorithm. I understand that you've read the introductory books on AI and ML, but I think that the online classes will provide you with a lot of exercises that you can try.
Although this is not an exact match to what you are looking for, textgraphs is a workshop that focuses on the link between graph theory and natural language processing. Here is a link. I believe the workshop also generated this book.