What is Snaplogic? - etl

As per Wikipedia:
SnapLogic is a commercial software company that provides Integration Platform as a Service (iPaaS) tools for connecting Cloud data sources, SaaS applications and on-premises business software applications.
It is surely a competitor to informatica, but it doesn't seem to be just another ETL tool. I have a rough understanding that it is used for data integration but that's about it.
Is it merely an ETL tool or does it have any other functionality? Also, what are iPaaS tools in general?

Well, the best place to learn about SnapLogic will be their website, https://www.snaplogic.com/
Here is a video of SnapLogic: https://www.youtube.com/watch?v=KYJK7bjOlA0
A simple developer friendly example:
Let's say i want to search for twitter feeds posted with a particular hashtag by a particular person and write that data into a database of my choice or into amazon S3. SnapLogic allows me to do that without learning about the Twitter API and AWS. SnapLogic takes care of the abstraction for the user so that they can focus on the business logic of things.
The demos are available in the blog: http://www.snaplogic.com/blog
A look at SnapLogic on crunchbase is not a bad idea and you could also find their competitors there.
Your other questions like what are integration platform as a service(iPaaS) tools is too basic and should just be googled.
Basically there are two types of cloud integration, iPaaS and dPaaS

Basically SnapLogic is a iPaas Tool (Integration Platforma as a Service), One of the growing online cloud based integration ETL tool.
As per the Gartner report one of leader in "Enterprise Integration Platform as a Service" refer below link
https://www.snaplogic.com/press-releases/gartner-names-snaplogic-as-a-leader-in-the-magic-quadrant
About tool and few useful links,
1.Designer
It's a canvas area to play and develop your integration pipelines.
2.Manager
Is is a main important to manage all the projects , pipelines, assets ,accounts,creating users and providing permissions, and also import and export of
projects.
3.Dashboard
Monitoring the pipelines logs and information of running pipelines and history.
Useful links:
1.Main site:
https://www.snaplogic.com
2.SnapLogic free trial:
https://www.snaplogic.com/free-trial
3.SnapLogic Documentation
http://doc.snaplogic.com/
4.SnapLogic Community
https://community.snaplogic.com/
5.SnapLogic Blog
https://www.snaplogic.com/blog
Also recently they released SnapLogic Extremem, Please have a look
https://www.snaplogic.com/press-releases/introducing-snaplogic-extreme-to-help-data-engineers-operationalize-cloud-based-big-data-integrations

Related

Using custom metrics in self-hosted sentry

I have started using sentry within my org and loving it so far.
I've been trying to use its performance monitoring tool with custom metrics added.
While I can add custom metrics to the transactions I'm generating in sentry_sdk (for Python), I can't get access to them on the dashboard of our self-hosted installation of sentry.
After a lot of digging, I came across this paragraph here which states that
This feature is only available to organization on our latest plans which include Dynamic Sampling. Customers on legacy plans must move to one of these plans in order to access custom metrics.
From what I gather, I believe their plans in general is to run sentry on their servers. Unless you opt-in to their self-hosted code that can be downloaded from github here.
This is absolutely a bummer because I know my org will not consider moving internal data to third-party servers.
Wondering if someone knows of a solution to this problem. If sentry folks know of (paid) options that enables this feature on self-hosted version or if someone has hacked into their open source code?
I'd also love to hear any out-of-the-box suggestion you folks might have.

Ideas to divided bandwidth to clients in Google Cloud Platform

How to separate bandwidth to clients in SaaS plaform?
Example: wix separate in GB to each client type.
I use Laravel in Google compute engine, but any tip is valid
I believe If we talking about SaaS your have to find your solution from third-party tools since GCP mostly is a Infrastructure as a Service (IaaS) or Platform as a Service (PaaS).
Please take look at following document which outline Rate-limiting strategies and techniques in GCP
https://cloud.google.com/solutions/rate-limiting-strategies-techniques
and
https://stripe.com/blog/rate-limiters

Can you use Microsoft Power Automate to develop a company wide workflow solution?

I am currently investigating possible tools with which to develop a workflow solution for a company that would be used by the staff to process incoming work and ensure incoming work is routed to the correct people for evaluation and are authorised by the relevant managers before work commences, and that the work is signed off when complete.
Is Microsoft Power Automate the right tool for this kind of work, or is it [simply] a tool for automating ones own personal tasks? I'm looking for something that could be used to develop a tool that would be used by everyone in the company.
I've looked at the various videos but it looks like it might be just something for automating ones personal tasks rather than building something to be shared. Is this correct or can it be used as a software development tool for developing a custom solution for a company?
Look at Business Process Flows (BPFs).
https://learn.microsoft.com/en-us/power-automate/business-process-flows-overview
The following text is from "Why use business process flows?" on the above link:
Business process flows provide a guide for people to get work done.
They provide a streamlined user experience that leads people through
the processes their organization has defined for interactions that
need to be advanced to a conclusion of some kind. This user experience
can be tailored so that people with different security roles can have
an experience that best suits the work they do.
Use business process flows to define a set of steps for people to
follow to take them to a desired outcome. These steps provide a visual
indicator that tells people where they are in the business process.
Business process flows reduce the need for training because new users
don’t have to focus on which table they should be using. They can let
the process guide them. You can configure business process flows to
support common sales methodologies that can help your sales groups
achieve better results. For service groups, business process flows can
help new staff get up-to-speed more quickly and avoid mistakes that
could result in unsatisfied customers.
You can create broad, or detailed, BPFs, to cover company wide business processes. You can also call on separate normal flows, validations, and more. There are a lot of ways you can use a BPF to move things forward in a company.
Power Automate (aka MS Flow) is a low code/no code business workflow management or process orchestration software, getting more power with Robotic Process Automation & Virtual Agents inclusion.
The concept is having variety of connectors for all the available SaaS solutions (name anything like Dynamics, Salesforce, Oracle, SAP, Twitter, Facebook, Exchange mailbox, OneDrive, SharePoint, etc) to act as both Trigger and Action to achieve what we want. At worst case we can use REST APIs to complete the project setup.
Community demo the usefulness of it by showcasing the automation of personal stuffs but it has more. MS Flow is the alternate for Dynamics CRM 365 workflow and SharePoint workflow as well. Definitely it’s an enterprise candidate to solve company wide problems like onboarding, approvals, and many other workflow solutions.
Personally I would look at Azure Logic Apps for that level. It has just about the same connectors as MS Flow , but with the addition of better management, troubleshooting and extensibility capabilities and you get all the benefits of security in Azure and compliance good stuff that companies would need

LUIS Offline Support

This might be a basic query but I wanted to confirm if there is any way of using the LUIS services/functionality offline without consuming the online APIs and generating the public key online. By offline, I mean to say if there are any supporting libraries/DLLs available for on-premise solutions by which we can build the Intends/Utterences/Entities and train the NLP system offline.
(The reason for asking this question is that I want to integrate LUIS with my existing Microsoft Bot application. However, our organization limits the software usage to utilize only on-premise offline software and any online software/services/APIs such as Azure APIs are restricted.)
Assuming that there is no such offline support for LUIS AI, are there any other libraries that would provide such support for .NET apps. I have come across Apache OpenNLP but that seems to more of Java-oriented offering.
Any inputs/suggestions on this would be appreciated.
Luis now has the ability to be fully off-cloud and on-premise through a docker container pull. This would be available for on-premise and Azure's IoT Edge (intelligent Edge) products.
Keep in mind
The described solution still requires a connection for Azure LUIS billing purposes that at the time of this writing is at a 15 minute interval. I believe this will be adjusted at some point in the future but it is something to keep in mind and plan for.
Link to Container Support in Azure Cognitive Services
As well, specifically, LUIS has full integration via this scenario. Where you can install and run LUIS docker containers. Please, keep in mind at the time of this writing this is ONLY for LUIS. Not QNA, not Bing Spell Check, *not analytics from the LUIS endpoints but there is a container work around for that.
Also, not speech priming or sentiment analysis and a few other features listed in the article.
Lastly, you can look into "Azure Stack" and IoT Edge here
It's a promising start. There are work arounds you may have to engineer while these services bare out more fruit but keep asking questions and keep raising concerns and more features will surely come!
LUIS is based on an online use, like almost all Microsoft Cognitive Services (except Custom vision compact models for example). There is no possibility for offline use, even if it may be useful for some cases like mobile use also.
Moreover (see here in the official documentation):
Is LUIS available on-premises or in private cloud? => No
For your 2nd question, StackOverflow may not be the right place for this (see https://stackoverflow.com/help/on-topic) and... I don't have a good solution! But would be interested to know one

Is Parse an adequate solution here?

I'm contemplating to use Parse as a platform for my app, as I'm trying to avoid creating and managing the cloud infrastructure myself.
For the sake of simplicity let's say that my app will hook into an Exchange Server and will need to leverage some hosted Machine Learning service to categorize my e-mail and report on insights found.
I'm assuming that Parse would store my core data, while the hosted ML will store the "Big Data" associated with processing for insights.
I'm also expecting my app to receive push notifications generated by the hosted ML service.
Does this sound like a plausible way to go about it and leverage Parse, or am I better off developing the backend myself?
I think parse.com is the right place for you requirements, because they have everything you need like storage of core data, push notifications, cloud module which can be integrated with heroku, social integration, user management functionalities.
They also have large set of client libraries for desktop and mobile apps (node,java,.net etc...) also they have libraries of embedded devices.
The biggest advantage is that everything is setup, and you are focused on software development not on infrastructure things. This is my opinion.
I've been experimenting with the above stack and so far was really impressed. Seems like a viable path forward. The Cloud Code capability of Parse is very solid, and easy to work with. If you want to run services outside of Parse code this us also possible : just issue REST calls.

Resources