I've found an issue and tracked it down to url conf. I'm attempting to perform an ajax post to the /gallery/add page which adds a new record into the database.
Originally I added a urls.py into my app and then 'include'ed it from the root urls.py but that failed during the ajax post (appears /gallery/ is just returned from logging).
Then i reverted to just the root urls.py and it worked as i expected.
So the question is are these urlconfs equivalent?
(A)
# ./urls.py
from django.conf.urls.defaults import *
urlpatterns = patterns('',
(r'^gallery$', 'gallery.views.home'),
(r'^gallery/add$', 'gallery.views.add'), # ajax post works with this one
)
(B)
# ./urls.py
from django.conf.urls.defaults import *
urlpatterns = patterns('',
(r'^gallery/', include('gallery.urls')),
)
# ./gallery/urls.py
from django.conf.urls.defaults import *
urlpatterns = patterns('',
(r'$', 'gallery.views.home'),
(r'add$', 'gallery.views.add'), # ajax request doesn't work, instead it goes to gallery.views.home
)
In the second example you still need the ^ because otherwise the first regex will just match any old string that has an ending (due to the $), and that is of course all of them :)
# ./gallery/urls.py
from django.conf.urls.defaults import *
urlpatterns = patterns('',
(r'^$', 'gallery.views.home'),
(r'^add$', 'gallery.views.add'),
)
Related
solution from #brian-destura below
The DRF test client does not work, but the django.test.client does. Odd (?) because it's a DRF APIView being called.
from django.test import Client
client = Client()
result = client.get('/api/place/6873947')
print(result.json)
I have a DRF DetailAPIView() that returns a complex serializer json response to external API queries, so in the browser, and via curl etc. http://localhost:8000/api/place/6873947/ returns a big JSON object. All good. The url entry in the 'api' app looks like this
path('place/<int:pk>/', views.PlaceDetailAPIView.as_view(), name='place-detail'),
I need to use that in another, function-based view, so first I tried using both django.test.Client and rest_framework.test.APIClient, e.g.
from rest_framework.test import APIClient
from django.urls import reverse
client = APIClient()
url = '/api/place/6873947/'
res = client.get(url)
That gets an empty result. With django Client:
from django.test import Client
c=Client()
Then
res = c.get('/api/place?pk=6873947')
and
res = c.get('/api/place/', {'pk': 6873947})
Both return "as_view() takes 1 positional argument but 2 were given"
I've tried other approaches in my IDE, picked up in StackOverflow, starting with
from api.views import PlaceDetailAPIView
pid = 6873947
from django.test import Client
from django.http import HttpRequest
from places.models import Place
request = HttpRequest()
request.method='GET'
request.GET = {"pk": pid}
Then
res = PlaceDetailAPIView.as_view({"pk": pid})
"as_view() takes 1 positional argument but 2 were given"
res = PlaceDetailAPIView.as_view()(request=request)
"Expected view PlaceDetailAPIView to be called with a URL keyword argument named "pk". Fix your URL conf, or set the .lookup_field attribute on the view correctly"
res = PlaceDetailAPIView.as_view()(request=request._request)
"HttpRequest' object has no attribute '_request"
I must be missing something basic, but hours of thrashing has gotten me nowhere - ideas?
When I have an urls.py file like this:
# urls.py
router = SimpleRouter(trailing_slash=False)
router.register("/?", MembersController, basename="member")
urlpatterns = router.urls
Then the generated URL for the single object is (?P<pk>[^/.]+)$. I'd like for it to include the int: "converter type". Is that possible at all? Or would I have to stop using DRF's router and create my own URL patterns?
In your MembersController you can specify the '[0-9]+' as lookup_value_regex:
class MembersController(ModelViewSet):
lookup_value_regex = '[0-9]+'
# ⋮
as default it makes use of '[^/.]+', as we can see in the source code [GitHub]:
lookup_value = getattr(viewset, 'lookup_value_regex', '[^/.]+')
cronjob
*/30 * * * * /home/ubuntu/web-coin-crawler/venv/bin/python3 /home/ubuntu/web-coin-crawler/webcoincrawler/cron.crontab
cron.py, crontab() Function that crawls on two sites. Save to Django orm as save().
import json
import collections
import crawl_coinmarketcal as coinmarketcal
import crawl_coinscalendar as coinscalendar
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
import django
import datetime
django.setup()
from crawled_data.models import BlogData
def preprocessingDict(dic: dict):
coin_dict = collections.defaultdict(dict)
for key, value in dic.items():
if value['symbol'] in coin_dict[value['date']]:
coin_dict[value['date']][value['symbol']].append([key, value['title'], value['name']])
else:
coin_dict[value['date']][value['symbol']] = [[key, value['title'], value['name']]]
return coin_dict
def crontab():
result = dict()
urls = coinmarketcal.get_urls()
for url in urls:
coinmarketcal.do_crawl(url, result)
urls = coinscalendar.get_urls()
for url in urls:
coinscalendar.do_crawl(url, result)
BlogData(title="COIN_DATA", content=json.dumps(preprocessingDict(result.copy()))).save()
I checked that the cron tab runs every 30 minutes.
However, no data was stored in Django orm.
I changed the Python file itself to run and it was solved.
cron.crontab -> cron
Below is the code for scraping emails for a single base url and I have been cracking my head on getting a simple "for loop" to do it for an array of urls or reading a list of urls (csv) into python. Can anyone modify the code so that it will do the job?
import requests
import re
from bs4 import BeautifulSoup
allLinks = [];mails=[]
url = 'https://www.smu.edu.sg/'
response = requests.get(url)
soup=BeautifulSoup(response.text,'html.parser')
links = [a.attrs.get('href') for a in soup.select('a[href]') ]
allLinks=set(links)
def findMails(soup):
for name in soup.find_all('a'):
if(name is not None):
emailText=name.text
match=bool(re.match('[a-zA-Z0-9_.+-]+#[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$',emailText))
if('#' in emailText and match==True):
emailText=emailText.replace(" ",'').replace('\r','')
emailText=emailText.replace('\n','').replace('\t','')
if(len(mails)==0)or(emailText not in mails):
print(emailText)
mails.append(emailText)
for link in allLinks:
if(link.startswith("http") or link.startswith("www")):
r=requests.get(link)
data=r.text
soup=BeautifulSoup(data,'html.parser')
findMails(soup)
else:
newurl=url+link
r=requests.get(newurl)
data=r.text
soup=BeautifulSoup(data,'html.parser')
findMails(soup)
mails=set(mails)
if(len(mails)==0):
print("NO MAILS FOUND")
I'm stuck in trying to create a simplest application with dajaxice.
I'n read all topics about this problem here and not only here, rewrite all code many times, but still do not see what the problem is.
the most interesting that, these examples are working (almost all):
https://github.com/jorgebastida/django-dajaxice/downloads dajaxice-examples.tar.gz
But in my project i have this:
Uncaught TypeError: Cannot call method 'sayhello' of undefined
my tools:
Windows 7 64
python-2.7.3
Django-1.4.2
django-dajaxice-0.2
project structure:
BlocalProject/
templates/
template_1.html
manage.py
BlocalProject/
ajapp/
__init__.py
ajview.py
__init__.py
settings.py
urls.py
views.py
wsgi.py
urls.py:
from django.conf.urls.defaults import *
import settings
from dajaxice.core import dajaxice_autodiscover
dajaxice_autodiscover()
urlpatterns = patterns('',
(r'^%s/' % (settings.DAJAXICE_MEDIA_PREFIX), include('dajaxice.urls')),
(r'^$', 'BlocalProject.views.start_page'),
)
views.py:
from django.shortcuts import render
def start_page(request):
return render(request,'template_1.html')
ajapp.py:
from django.utils import simplejson
from dajaxice.core import dajaxice_functions
def sayhello(request):
return simplejson.dumps({'message': 'Trololo!'})
dajaxice_functions.register(sayhello)
template_1.html:
{% load dajaxice_templatetags %}
<html>
{% dajaxice_js_import %}
<script>
function alertMessage(data){
alert(data.message);
return false;
}
</script>
<body>
Some text
<input type="button" value="Get!" onclick="Dajaxice.ajapp.sayhello(alertMessage);" />
</body>
</html>
settings.py:
# Django settings for BlocalProject project.
DEBUG = True
TEMPLATE_DEBUG = DEBUG
ADMINS = (
# ('Your Name', 'your_email#example.com'),
)
MANAGERS = ADMINS
DATABASE_ENGINE = '' # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
DATABASE_NAME = '' # Or path to database file if using sqlite3.
DATABASE_USER = '' # Not used with sqlite3.
DATABASE_PASSWORD = '' # Not used with sqlite3.
DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3.
# Local time zone for this installation. Choices can be found here:
# http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
# although not all choices may be available on all operating systems.
# In a Windows environment this must be set to your system time zone.
TIME_ZONE = 'America/Chicago'
# Language code for this installation. All choices can be found here:
# http://www.i18nguy.com/unicode/language-identifiers.html
LANGUAGE_CODE = 'en-us'
SITE_ID = 1
# If you set this to False, Django will make some optimizations so as not
# to load the internationalization machinery.
USE_I18N = True
# Absolute filesystem path to the directory that will hold user-uploaded files.
# Example: "/home/media/media.lawrence.com/media/"
MEDIA_ROOT = ''
# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash.
# Examples: "http://media.lawrence.com/media/", "http://example.com/media/"
MEDIA_URL = ''
# URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a
# trailing slash.
# Examples: "http://foo.com/media/", "/media/".
ADMIN_MEDIA_PREFIX = '/media/'
# URL prefix for static files.
# Example: "http://media.lawrence.com/static/"
STATIC_URL = '/'
# Make this unique, and don't share it with anybody.
SECRET_KEY = ')er9!%4v0=nmxd#2=j1*tlktmidq8aam2y)-%fjf6%^xp*5r)c'
# List of callables that know how to import templates from various sources.
TEMPLATE_LOADERS = (
'django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader',
#'django.template.loaders.eggs.load_template_source',
)
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
)
ROOT_URLCONF = 'BlocalProject.urls'
# Python dotted path to the WSGI application used by Django's runserver.
#WSGI_APPLICATION = 'BlocalProject.wsgi.application'
TEMPLATE_DIRS = (
'D:/_/Site_test/Djpr/BlocalProject/templates',
# Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
# Always use forward slashes, even on Windows.
# Don't forget to use absolute paths, not relative paths.
)
TEMPLATE_CONTEXT_PROCESSORS = ("django.contrib.auth.context_processors.auth",
"django.core.context_processors.debug",
"django.core.context_processors.i18n",
"django.core.context_processors.media",
"django.core.context_processors.static",
"django.core.context_processors.request",
"django.contrib.messages.context_processors.messages",)
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'dajaxice',
'BlocalProject.ajapp',
)
DAJAXICE_MEDIA_PREFIX = "dajaxice"
DAJAXICE_DEBUG = True
DAJAXICE_JS_DOCSTRINGS = True
#DAJAXICE_NOTIFY_EXCEPTIONS = True
import logging
logging.basicConfig(level=logging.DEBUG)
This might be too late, but...
IMHO, the dajaxice_autodiscover() doesn't actually get your method. I remember having a similar problem with another ajax lib and I've solved it by adding an import in the models.py (or views.py), which get imported when the app starts. The example you mention has an import in simple/views.py:
# Create your views here.
from django.shortcuts import render
# THIS ONE!
from dajaxice.core import dajaxice_functions
# ---------
def index(request):
return render(request, 'simple/index.html')
which looks like it's initialising stuff. My approach would be:
put a couple of print statements in your ajapp.py and see if they get printed out.
create empty models.py and views.py in your BlocalProject.ajapp (does django still use models.py to validate a module as a django app?)
If your print statements don't get triggered, then you need to find out why. As I mentioned, it might be as simple as importing your ajax module in models :)