RPCRequests and datastore - ajax

I'm getting started with Google App Engine. I want to make an AJAX chat.
class ChatMessage(db.Model):
message = db.StringProperty()
created = db.DateTimeProperty(auto_now_add=True)
class RPCHandler(webapp.RequestHandler):
##get message every 4 seconds
def get(self):
que = db.Query(ChatMessage).order('-created')
chat_list = que.fetch(limit=3)
jencoded_chat_list = gaejsonEncoder.encode(chat_list) ##my own module
self.response.out.write(jencoded_chat_list)
RESULT: message3,message2,message1
This is what I intended.
I wanted to add POST ajax Request, so added RPCHandler2.
class RPCHandler2(webapp.RequestHandler):
def post(self):
msg = self.request.get('message')
if msg == '':
return
newchat = ChatMessage(message=msg)
newchat.put()
que = db.Query(ChatMessage).order('-created')
chat_list = que.fetch(limit=3))
jencoded_chat_list = gaejsonEncoder.encode(chat_list)
self.response.out.write(jencoded_chat_list)
I write "POST Message!" in textarea and click the button,
RESULT: POST Message!,message3,message2
This is what I wanted.
But 4 seconds later, 'GET' started and
RESULT: message3,message2,message1
Why 'GET' couldn't get the new message from datastore?
Thanks in advance.

If you are using the high replication datastore you might be facing an "eventual consistency" problem. The datastore replica handling the get request may not have received the recently submitted data yet.
If you put the newchat.put() inside of a transaction it's more likely that future gets will be consistent.

Related

Code is not delievered by SendCodeRequest without error and this happens only on server-side with Heroku

Here is my situation: the same Telethon code is used on my local machine and on the server. Requesting authorization code from local machine works fine. Requesting the code from the server does not produce any error, and code is not sent. Sometimes it works even from the server without any changes in code.
I suppose there might be some ip blocks or something related to the ip, cause that is the only thing which might differ on the server side: Heroku assign ip addresses dynamically, so, there might by some subnets which are blocked by Telegram API for some reason. But there is no error and that is really strange. There are too many ip addresses to disprove the hypothesis. I need to catch at least one ip address which gives me opposite results: one time code it recieved and another time does not. So I am stuck with this situation and have no ideas how it could be fixed or clarified.
global t
t = None
async def ssssendCode(phone):
global t
try:
if os.path.isfile(phone+'.session'):
logger.debug('client file exists')
else:
logger.debug('client file does not exist')
if t is None:
t = TelegramClient(phone, settings['telegramClientAPIId'], settings['telegramClientAPIHash'])
t.phone = phone
#t.phone_code_hash = None
await t.connect()
#response = await t.send_code_request(phone=phone,force_sms=True)
s3_session.resource('s3').Bucket('telethon').upload_file(str(phone)+".session", str(phone)+".session")
logger.debug(str(requests.get('https://httpbin.org/ip').text))
response = await t.send_code_request(phone=phone)
logger.debug(str(t.is_connected()))
except Exception as e:
response = str(e)
return str(response)
example of response to the local machine request
SentCode(type=SentCodeTypeSms(length=5), phone_code_hash='b5b069a2a4122040f1', next_type=CodeTypeCall(), timeout=120)
example of reponse to the server-side request
SentCode(type=SentCodeTypeSms(length=5), phone_code_hash='0e89db0324c1af0149', next_type=CodeTypeCall(), timeout=120)
send_code_request is the from the Telethon without modifications
async def send_code_request(
self: 'TelegramClient',
phone: str,
*,
force_sms: bool = False) -> 'types.auth.SentCode':
"""
Sends the Telegram code needed to login to the given phone number.
Arguments
phone (`str` | `int`):
The phone to which the code will be sent.
force_sms (`bool`, optional):
Whether to force sending as SMS.
Returns
An instance of :tl:`SentCode`.
Example
.. code-block:: python
phone = '+34 123 123 123'
sent = await client.send_code_request(phone)
print(sent)
"""
result = None
phone = utils.parse_phone(phone) or self._phone
phone_hash = self._phone_code_hash.get(phone)
if not phone_hash:
try:
result = await self(functions.auth.SendCodeRequest(
phone, self.api_id, self.api_hash, types.CodeSettings()))
except errors.AuthRestartError:
return await self.send_code_request(phone, force_sms=force_sms)
# If we already sent a SMS, do not resend the code (hash may be empty)
if isinstance(result.type, types.auth.SentCodeTypeSms):
force_sms = False
# phone_code_hash may be empty, if it is, do not save it (#1283)
if result.phone_code_hash:
self._phone_code_hash[phone] = phone_hash = result.phone_code_hash
else:
force_sms = True
self._phone = phone
if force_sms:
result = await self(
functions.auth.ResendCodeRequest(phone, phone_hash))
self._phone_code_hash[phone] = result.phone_code_hash
return result
Just in case: I have much more than 2 minutes between attempts to get a code from the local machine and server, so it is absolutely not the timeout issue. And moreover: even when requesting the code from the local right after half a minute from the failed server-side attemp: code is coming almost immediately.

cron job with php-ews to get only new emails from Exchange server

I need a cron job to obtain only the new emails received in a Exchange server since the last time it synchronized.
I coded the same with imap servers, and it was easy to get the emails since an ID. So I save the last ID at the end of the for cycle to the DB, and resume from that ID the next time the cron job executes.
I tried the same in Exchange, but I get the following error:
Failed to search for messages with "ErrorInvalidValueForProperty: El valor especificado no es válido para la propiedad."
To get the ItemId in Exchange I'm using the following code, with no error:
$item->ItemId->Id;
It returns something like:
AAMkADk3ZWRhY2VmLTdiY2UtNDkxYy04ZDMyLWZiZWIyYTQ4ZjQyMABGAAAAAAChH5wDkXAqTZaoH1oRodz8BwD8BvGliMkPTK/cnnmZJDPUAAAAvGXfAAD8BvGliMkPTK/cnnmZJDPUAAAAvJN5AAA=
To find the emails I'm trying the following code (with the error mentioned before):
$request = new FindItemType();
$request->ParentFolderIds = new NonEmptyArrayOfBaseFolderIdsType();
$request->Traversal = ItemQueryTraversalType::SHALLOW;
$greater_than = new IsGreaterThanOrEqualToType();
$greater_than->FieldURI = new PathToUnindexedFieldType();
$greater_than->FieldURI->FieldURI = UnindexedFieldURIType::ITEM_ID;
$greater_than->FieldURIOrConstant = new FieldURIOrConstantType();
$greater_than->FieldURIOrConstant->Constant = new ConstantValueType();
$greater_than->FieldURIOrConstant->Constant->Value = $UID;
Thanks!!
The EWS ItemId isn't a searchable property because it doesn't need to be eg you can just Bind to the Id in question to see if the Item exists. If it nolonger exists then you will get a specific error to tell you that. The SyncFolderItems operation is probably a more appropriate method to use however https://learn.microsoft.com/en-us/exchange/client-developer/web-service-reference/syncfolderitems-operation.

Tornado cancel httpclient.AsyncHTTPClient fetch() from on_chunk()

Inside one of the handlers I am doing the following:
async def get(self):
client = httpclient.AsyncHTTPClient()
url = 'some url here'
request = httpclient.HTTPRequest(url=url, streaming_callback=self.on_chunk, request_timeout=120)
result = await client.fetch(request)
self.write("done")
#gen.coroutine
def on_chunk(self, chunk):
self.write(chunk)
yield self.flush()
The requests can sometimes be quite large and the client may leave while the request is still in progress of being fetched and pumped to the client. If this happens an exception will appear in the on_chunk function when self.write() is attempted. My question is how do I abort the remaining download if my client went away ?
If your streaming_callback raises an exception, the client request should be aborted. This will spam the logs with stack traces, but there's not currently a cleaner way to do it. You can override on_connection_close to detect when the client has disconnected and set an attribute on self that you can check in on_chunk.

App engine call to Google API python client return 403 with #oauth_required

It should be a trivial job but i cannot get it out.
I need to call the Google Calendar API from Gae; thus I set all up as per Google docs and examples:
I've an /auth.py:
CLIENT_SECRETS = os.path.join(os.path.dirname(__file__), 'client_secrets.json')
SCOPES = [
'https://www.googleapis.com/auth/calendar',
]
decorator = appengine.OAuth2DecoratorFromClientSecrets(
filename=CLIENT_SECRETS,
scope=SCOPES,
cache=memcache,
prompt='consent',
)
called by main.py functions:
class Landing(webapp2.RequestHandler):
#auth.decorator.oauth_aware
def get(self):
if auth.decorator.has_credentials():
self.redirect('/in')
else:
self.response.out.write('''
etc. {}'''.format(auth.decorator.authorize_url()))
class Main(webapp2.RequestHandler):
#auth.decorator.oauth_required
def get(self):
links = { ... }
render(self, 'base.html', template_values=links)
class Calendar(webapp2.RequestHandler):
#auth.decorator.oauth_required
def get(self):
service = build('calendar', 'v3', http=auth.decorator.http())
api_request = service.events().list(calendarId='primary')
api_response = api_request.execute()
self.response.headers['Content-Type'] = 'application/json; charset=utf-8'
self.response.out.write(json.dumps(api_response, indent=4))
class PutEvent(webapp2.RequestHandler):
#auth.decorator.oauth_required
def post(self):
# ...
# http = httplib2.Http(memcache)
service = build('calendar', 'v3') #, http=http)
api_response = []
for i in json.loads(self.request.get('events')):
# ...
event = { ... } # Google Calendar event
api_request = service.events().insert(calendarId='primary', body=scadenza)
api_response.append(api_request.execute(http=auth.decorator.http()))
self.response.headers['Content-Type'] = 'application/json; charset=utf-8'
self.response.out.write(json.dumps(api_response, indent=4))
As you can see this is a fairly simple post requested by an Ajax jQuery call ($.post('{{ putEvent_url }}', jsonData, function( data ){ console.log(data); })...
I'm in the development server, using test#example user, and the app is authorized to access my personal account's Google Calendar.
Strange thing to me is that any call to Calendar() works as expected, but call to PutEvent() end in ERROR 500.
Looking to the end of the traceback in console:
File "/home/pierpaolo/Devnos/whiterabbit/include/oauth2client/contrib/appengine.py", line 644, in check_oauth
resp = method(request_handler, *args, **kwargs)
File "/home/pierpaolo/Devnos/whiterabbit/main.py", line 211, in post
api_response.append(api_request.execute(http=auth.decorator.http()))
File "/home/pierpaolo/Devnos/whiterabbit/include/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/pierpaolo/Devnos/whiterabbit/include/googleapiclient/http.py", line 838, in execute
raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 403 when requesting https://www.googleapis.com/calendar/v3/calendars/primary/events?alt=json returned "Forbidden">
INFO 2017-01-04 15:13:32,385 module.py:788] default: "POST /api/put/scadenze HTTP/1.1" 500 -
I cannot understand the
HttpError: https://www.googleapis.com/calendar/v3/calendars/primary/events?alt=json returned "Forbidden">
it looks to me I already granted the app access to my account and that Google App Engine decorators have been correctly put in place to make the OAuth2.0 thing as per https://developers.google.com/api-client-library/python/guide/google_app_engine...
EDIT:
I was wondering if my trouble can be related to the way i call Google Calendar API:
HTML/JS GAE/Py
+------------------+
| |
| <form> |
| ...data |
| <\JS/Ajax |
| $.post(...data | --> GAE/main.py
| | #auth.decorator.oauth_required
| | def post(self, data):
+------------------+ event = elaborate(data)
service = build('calendar', 'v3')
api_request = service.events()
.insert(calendarId='primary',
body=event)
api_response = api_request
.execute(auth.decorator
.http())
self.response(api_response)
EDIT 3:
I looked a bit into oauth2client.contrib.appengine and I added some logger.debug here and there: I think the problem could be in execute(http=decorator.http()) call, but it is the same in my other handlers! Neither positional nor keyword nor put authrized Http in service build changes the misbehaviour...
Nor can I see what problem may pose _helpers.py", line 133, in positional_wrapper...
Dear all, some hint on how to research further?
Actually, I can insert Acl and/or insert a secondary calendar in the same RequestHandler that throws Forbidden exception with events().insert()...!
I haven't got enough reputation to comment so I'll just leave it as an answer:
Firstly, double check that you've got calandar api enabled at https://console.cloud.google.com/apis/dashboard?project=yourproject
Secondly. I've used the contacts API and discovered that once you've granted access once you cannot be granted access again unless you first revoke the initial allowance. I encountered this when a user connected my app to Google Contacts, then disconnected, then tried to reconnect - the second reconnect would fail. To check / revoke, head to https://security.google.com/settings/security/permissions
Apparently, the problem is to try to insert an all-day event with endTimeUnspecified: True...
I opened an issue on google-api-python-client GitHub tracker: https://github.com/google/google-api-python-client/issues/334.
Maybe someone will look into it or post a more precise answer.
Thank you all.

Reading Withings API ruby

I have been trying for days to pull down activity data from the Withings API using the OAuth Ruby gem. Regardless of what method I try I consistently get back a 503 error response (not enough params) even though I copied the example URI from the documentation, having of course swapped out the userid. Has anybody had any luck with this in the past. I hope it is just something stupid I am doing.
class Withings
API_KEY = 'REMOVED'
API_SECRET = 'REMOVED'
CONFIGURATION = { site: 'https://oauth.withings.com', request_token_path: '/account/request_token',
access_token_path: '/account/access_token', authorize_path: '/account/authorize' }
before do
#consumer = OAuth::Consumer.new API_KEY, API_SECRET, CONFIGURATION
#base_url ||= "#{request.env['rack.url_scheme']}://#{request.env['HTTP_HOST']}#{request.env['SCRIPT_NAME']}"
end
get '/' do
#request_token = #consumer.get_request_token oauth_callback: "#{#base_url}/access_token"
session[:token] = #request_token.token
session[:secret] = #request_token.secret
redirect #request_token.authorize_url
end
get '/access_token' do
#request_token = OAuth::RequestToken.new #consumer, session[:token], session[:secret]
#access_token = #request_token.get_access_token oauth_verifier: params[:oauth_verifier]
session[:token] = #access_token.token
session[:secret] = #access_token.secret
session[:userid] = params[:userid]
redirect "#{#base_url}/activity"
end
get '/activity' do
#access_token = OAuth::AccessToken.new #consumer, session[:token], session[:secret]
response = #access_token.get("http://wbsapi.withings.net/v2/measure?action=getactivity&userid=#{session[:userid]}&startdateymd=2014-01-01&enddateymd=2014-05-09")
JSON.parse(response.body)
end
end
For other API endpoints I get an error response of 247 - The userid provided is absent, or incorrect. This is really frustrating. Thanks
So I figured out the answer after copious amount of Googleing and grasping a better understanding of both the Withings API and the OAuth library I was using. Basically Withings uses query strings to pass in API parameters. I though I was going about passing these parameters correctly when I was making API calls, but apparently I needed to explicitly set the OAuth library to use the query string scheme, like so
http_method: :get, scheme: :query_string
This is appended to my OAuth consumer configuration and all worked fine immediately.

Resources