We've been developing a system that integrates with the GMail API via the Java SDK. The implementation works well when hosted outside of AWS. But when the same code is placed into our AWS EC2 instances, we receive socket timeouts (even with 6 minute timeouts) about 50% of the time. We have done extensive firewall, routing, and proxy checks - but have found nothing that should cause an issue. We integrate with many other similar APIs (including Google Drive) and none of them exhibit this problem. There are no errors shown in the GMail API developer console.
Is there some sort of rate limiter for AWS based GMail API clients?
Can someone from the GMail API team help me troubleshoot this connectivity issue?
Related
I am working on MS Teams development. I installed the MS Teams toolkit in VS Code, set up my subscription with Azure and sideloading is active in my tenant.
When I run the app, it tries to install ngrok. This step fails as my organization does not allow running ngrok or other words tunnelling from our company laptop. We can run this on a VM to go around this but VM is not always available.
I am looking for a resolution for below scenarios:
Is there a way to debug MS Teams application without ngrok?
If we need a https URL, is it possible to configure a web app to facilitate that?
I tried removing install ngrok step from: /.vscode/tasks.json, but there are subsequent steps it the file dependent on that
I've done quite a bit of research on this question myself as I'd been getting a lot of pushback from our IT department regarding the security threats that come with using a tunneling service like ngrok. It eventually led me to this video posted on the MS forums from a Microsoft engineer who explains it clearly.
What it comes down to is that the Teams client (browser/desktop) approaches webservices (configured in the manifest file) differently depending on the type of interaction. If you're testing configurable tabs, task modules or configuration pages, then you can easily route the app to those sites running on your localhost through the manifest. The Teams client will approach them directly. Problems start to arise when you want to debug what happens when you use a bot or message extension, outgoing webhook or MS Graph change notifications (just quoting the video here, there might be other scenarios).
Basically, what happens is that the Teams client goes through a Microsoft-hosted service first, called Microsoft Teams Services, which will then approach your bot framework cloud service (typically an Azure Bot resource). This then forwards any incoming messages to whatever endpoint you have configured. What happens in these separate stages isn't completely clear to me, but what I do know is that whatever is typed by the user in the Teams client is translated to a JSON structure that can be interpreted by your server-side bot code (for C# apps, this is typically your CloudAdapter-derived class working with your TeamsBot-derived class). These messages are then routed to the relevant TeamsBot class method based on properties in the JSON.
Now the issue that ngrok solves is that, when the Teams client goes onto the public internet to reach the MS Teams Services server and then the Azure Bot resource, it then needs a public address to route the traffic to. It doesn't know about your local network anymore. As ngrok sets up a TCP tunnel between their server and your local PC, it is able to route traffic coming to their server to your PC. The Azure Bot now has a public address to send the messages to.
To my knowledge, there is no way to circumvent this as long as Teams client inner workings always make it go outside of your local network. For chat scenarios, the Bot Framework Emulator might offer a solution for unit testing. As far as I can see it performs the translation of chat input to the JSON message model of the Bot Framework and routes it to a local address for your chatbot to process it. Unfortunately, this doesn't work for chat message extension type messages.
As for the question whether ngrok can be avoided, I think the answer is definitely yes but you would need an alternative. There's several alternatives around that you might be able to host yourself if you have the technical know-how. Depending on your IT department, being in control of the public-internet-facing server might be a more viable solution for them. Another option is to host ngrok on a VM or cloud machine with less access to your internal network's resources than your PC/laptop has and test the code there.
TL;DR: If the the feature you're testing is approached directly by the Teams client, you can enter localhost in the manifest and debug it. If you're testing a feature that the teams client approaches through Microsoft Teams Services and the Bot Framework, you need to find a way to expose your code to the public internet. You can use ngrok or host your own alternative depending on requirements.
use mkcert to generate a certificate for ex. localhost.test
add losthost.test to your host file
use https://localhost.test for debugging
Did anyone deploy Azue Web App Bot using Application Service Environment?
are there any key considerations to be noted? I know we have ASE ILB and ASE External;
Is it possible to host multiple azure webapp bots in one ASE; my question is primarily due to the default lockdown of internet traffice in ASE ILB model and what type of firewall exceptions we will need to ensure
functinally the communication to Azure Bot Service/ Directline happen smoothly.
It is possible to host multiple azure web app bots in one ASE. However, care should be taken on how to have the bot dynamically looks up the pipe name as there a multiple bots inside the same ASE. Also, the normal DirectLine or other channels would require a lot of whitelisting to allow traffic into the ILB, and bot services IPs can change so it would be difficult to maintain long term.
'Test in WebChat' is not expected to work within an ILB ASE. It calls out to the DirectLine channel and causes the channel to send a call to the bot's messaging endpoint. In most ASE or VNET scenarios that call will be blocked, but since we don't have static IP addresses so the customer can't whitelist the incoming calls, either. Other than that, Directline channel and Direct Line App Service Extension(DL ASE) should typically work as expected from within an ILB ASE setup. If you are implementing additional features such as OAuth or SSO, then you will need to add a rule to enable service tags for AzureAD.
For more info on DL ASE, please refer to https://learn.microsoft.com/en-us/azure/bot-service/bot-service-channel-directline-extension?view=azure-bot-service-4.0
I've been following Microsoft's Teams C# tutorials found here, and have been successful for the most part. However, I cannot seem to get my app to work when I host the messaging endpoint myself rather than via their Azure service, which is not an option for me ultimately as the pricing is outrageous for what we need it to do.
I'm hosting the endpoint myself by publishing the sample project and ensuring it's externally available via HTTPS. I can access a custom tab within Teams, so I know that it's online, it's just the messaging endpoint that seems to fail with an "unable to reach app" error when I try and use the messaging extension via a chat window.
When debugging using dev tools, I get 502 error: Bot returned unsuccessful status code Forbidden, error code 1008. Every potential solution I've seen for similar issues hasn't worked for me thus far, though I still feel like it's something incredibly obvious. Are there special steps that need taking when hosting the endpoint yourself? The docs do a very lousy job of explaining the process, probably because Microsoft want you to pay to host the app on Azure.
This is usually caused by the app id / app key not being registered or used correctly in your app, so it's not authenticating to the bot framework service properly. Where/how you do that depends a bit on what sample code / project template you started with, but it's usually somewhere in a .config file (or previously in a .bot file).
The information that you need will be in:
App Id: The Bot Settings page in Azure
App Key: from the Bot settings page, where you got the AppId above, it links to the App registration itself - within there you'll find the section on keys, and you can create a new key (if you've lost the original one)
I know it's generally an error when AppID validation fails. The bot app requests Azure AD to verify the identity.Could your web server access to Azure AD? If you deny to access to outbound with firewall, you should allow Azure IP range.
Turns out it was purely a network issue, that as of yet we still haven't actually figured out. But we tried hosting the app elsewhere and it was fine. That's my recommendation if anyone else has the same problem!
I am trying to test my bot. It is hosted remotely but on a server that has no internet connectivity but is reachable within the LAN. I want to use the Botframework emulator to test. The emulator relies on ngrok. But I have two restrictions :
1) I cannot install ngrok on my corporate machine.
2) Even if I somehow managed to cut through the red tape and install ngrok , without internet connectivity on the server that hosts my bot, the responses would still not reach my emulator.
How can I use a service url that hits my emulator directly without using ngrok ?
First, ngrok is only necessary for connecting to bots hosted remotely. If you are looking to run everything locally on the closed server, then ngrok is not required.
With regards to options, you have a couple that may work for you.
One, you can look at utilizing offline-directline. This option allows you to generate a token locally without having to connect to the public direct line offering on Azure. Be aware that this npm package is configured for the v3 "BotChat" web chat tool. So, utilizing this will require your modifying the configuration to work with the newer v4 Web Chat (not to be confused with the v3/v4 SDKs).
Two, consider using this Browser Bot sample, located here, from the Botbuilder-Samples GitHub repo. In this instance, the bot and web chat adapters are fully contained within the browser and, as such, do not require a connection to direct line to run. The bot adapter uses the v4 Node SDK while the web chat adapter uses the v4 React-based implementation.
Hope of help!
I am trying to integrate SES with Sendmail in My EC2 Server.
I looked this documents and followed steps, but didn't work out.
http://docs.amazonwebservices.com/ses/latest/DeveloperGuide/SMTP.MTAs.Sendmail.html
I hoped to see video, but there wasn't anything about this one.
I couldn't even find a good article or blog about this.
Can you help me out? I just need to set up Sendmail via SSH.
This is now possible -- log into your AWS Console, to the SES Dashboard, and you'll see a section titled "SMTP Settings" on the lefthand nav. There you will find instructions and a mechanism to create your own SMTP credentials.
At that point you will need to set up your Postfix or Sendmail to act as a relay for outbound mail, redirecting it to the AWS SES Service API.
HOWEVER a few caveats:
You won't be able to use this as some normal SMTP gateway or outbound mail host unless you were to verify any possible sender for mail headed out of your instance. A more ideal case for this is a vendor who sends out large marketing mails to its customers, or a service that pushes out mail on a subscription or alert basis, all of which come from just one or a small handful of addresses.
Also, using SES means you'll have to live by AWS's thresholds, even once you've requested production access. An AWS engineer told me in training that they generally keep an eye on your mailflow and any bounces or rejects, as well as complaints made about your mail, and so long as things look good they will continue to throttle you up in that regard.
Good luck to you.