I'm working with what I believe is a fairly standard FusionAuth installation. From a few days ago, admin users have been unable to access the Dashboard at [myhost]:[myport]/admin/ (aside from this, FusionAuth appears to be working as normal).
Once a user has entered their login credentials, the page becomes unresponsive for some time (typically several minutes), before presenting them with this (obviously generic) message:
enter image description here
The StdErr logs show the following:
Jun 08, 2020 3:05:34 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.primeframework.mvc.freemarker.FreeMarkerRenderException: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> loginData [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 46]
The relevant section of index.ftl is as follows:
[#layout.html]
[#layout.head]
<script>
Prime.Document.onReady(function() {
new FusionAuth.Admin.Dashboard([[#list loginData.labels as label]"${label}"[#sep], [/#list]],
[[#list loginData.counts as count]${count}[#sep], [/#list]]);
});
</script>
[/#layout.head]
...and the loginData variable is declared as follows:
[#-- #ftlvariable name="loginData" type="io.fusionauth.app.service.ReportUtil.ReportData" --]
If a user enters the wrong password, they get the usual failure message, with little delay; this issue seems to occur after initial authentication. I understand that loginData is likely to be a report, but am uncertain of its significance (and the standard FusionAuth login report seems to work OK when I call the API via Postman). Has anyone had a similar issue?
Edit: I'm advised that this is a standard installation, with Elasticsearch and MySQL as recommended. User count isn't believed to be unusual (totals report gives globalRegistrations and totalGlobalRegistrations in the 7500 to 8500 range).
Additional information:
Versions:
OS: Windows Server 2019 Standard, V1809
MySql: 8.0
ElasticSearch:6.3.1
FusionAuth: 1.9.1
There have been no known recent deliberate changes to this FusionAuth system
When the error occurs, no error were initially reported in the JS Console, but I've now seen one relating to a missing "]", and the reason for this is that the error message seen below appears to have been inserted into the page source, presumably while it was being generated using a template:
<script>
var FusionAuth = FusionAuth || {};
FusionAuth.requestContextPath = '';
FusionAuth.loginURI = FusionAuth.requestContextPath + '/login';
</script>
<script>
Prime.Document.onReady(function() {
new FusionAuth.Admin.Dashboard([FreeMarker template error (DEBUG mode; use RETHROW in production!):
The following has evaluated to null or missing:
==> loginData [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 46]
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use [#if myOptionalVar??]when-present[#else]when-missing[/#if]. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: #list loginData.labels as label [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 39]
~ Reached through: #nested [in template "WEB-INF/templates/_layouts/admin.ftl" in macro "head" at line 124, column 3]
~ Reached through: #layout.head [in template "WEB-INF/templates/admin/index.ftl" at line 20, column 1]
~ Reached through: #nested [in template "WEB-INF/templates/_layouts/admin.ftl" in macro "html" at line 14, column 3]
~ Reached through: #layout.html [in template "WEB-INF/templates/admin/index.ftl" at line 19, column 1]
----
Java stack trace (for programmers):
----
freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...]
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:134)
at freemarker.core.UnexpectedTypeException.newDesciptionBuilder(UnexpectedTypeException.java:85)
at freemarker.core.UnexpectedTypeException.<init>(UnexpectedTypeException.java:48)
at freemarker.core.NonHashException.<init>(NonHashException.java:49)
...
Possible cause: I do see some apparent MySQL timeouts, now that I come to check the StdOut logs rather than StdErr; maybe that's the issue; will confirm - scratch that; the latest case was not accompanied by any obvious MySQL-related log entries.
Latest full error message (some lines removed from stack trace to save space. This error was not preceded directly by anything in either the StdOut or StdErr logs; it was the day's first entry in the StdErr log):
Jun 11, 2020 9:55:13 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [default] in context with path [] threw exception
org.primeframework.mvc.freemarker.FreeMarkerRenderException: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> loginData [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 46]
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use [#if myOptionalVar??]when-present[#else]when-missing[/#if]. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: #list loginData.labels as label [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 39]
~ Reached through: #nested [in template "WEB-INF/templates/_layouts/admin.ftl" in macro "head" at line 124, column 3]
~ Reached through: #layout.head [in template "WEB-INF/templates/admin/index.ftl" at line 20, column 1]
~ Reached through: #nested [in template "WEB-INF/templates/_layouts/admin.ftl" in macro "html" at line 14, column 3]
~ Reached through: #layout.html [in template "WEB-INF/templates/admin/index.ftl" at line 19, column 1]
----
at org.primeframework.mvc.freemarker.DefaultFreeMarkerService.render(DefaultFreeMarkerService.java:69)
...
at io.fusionauth.app.primeframework.FusionAuthMVCWorkflow.perform(FusionAuthMVCWorkflow.java:93)
...
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> loginData [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 46]
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use [#if myOptionalVar??]when-present[#else]when-missing[/#if]. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: #list loginData.labels as label [in template "WEB-INF/templates/admin/index.ftl" at line 23, column 39]
~ Reached through: #nested [in template "WEB-INF/templates/_layouts/admin.ftl" in macro "head" at line 124, column 3]
~ Reached through: #layout.head [in template "WEB-INF/templates/admin/index.ftl" at line 20, column 1]
~ Reached through: #nested [in template "WEB-INF/templates/_layouts/admin.ftl" in macro "html" at line 14, column 3]
~ Reached through: #layout.html [in template "WEB-INF/templates/admin/index.ftl" at line 19, column 1]
----
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:134)
...
at org.primeframework.mvc.freemarker.DefaultFreeMarkerService.render(DefaultFreeMarkerService.java:65)
... 35 more
Work-around: If we go to, for example, [host:port]/admin/user (ie. specify a page other than the Dashboard), the actual admin functionality can be used as normal.
NB A similar issue occurs if we attempt to use [host:port]admin/system/login-record/
Possible query issue
We have notices that the following query - which is believed to be the one that pulls the login data involved in this issue - is running slowly and locking up database connections:
SELECT
rl.applications_id AS applicationId,
rl.instant AS instant,
rl.ip_address AS ipAddress,
rl.users_id AS userId,
a.name AS applicationName,
COALESCE(i.email, i.username) AS loginId
FROM raw_logins AS rl
INNER JOIN applications AS a ON a.id = rl.applications_id
INNER JOIN identities AS i ON i.users_id = rl.users_id
ORDER BY rl.instant DESC
LIMIT 5
OFFSET 0
Is this a standard FusionAuth query? Have performance issues been reported with it?
Related
I am using Sphinx 4.4.0 with napoleon extension (Google Doc String). I have this two problems
ARNING: Block quote ends without a blank line; unexpected unindent.
ERROR: Unexpected indentation.
I found something about it on the internet but can not fit this two my code. My problem is I even do not understand the messages. I do not see where the problem could be.
This is the code:
def read_and_validate_csv(basename, specs_and_rules):
"""Read a CSV file with respect to specifications about format and
rules about valid values.
Hints: Do not use objects of type type (e.g. str instead of "str") when
specificing the column type.
specs_and_rules = {
'TEMPLATES': {
'T1l': ('Int16', [-9, ' '])
},
'ColumnA': 'str',
'ColumnB': ('str', 'no answer'),
'ColumnC': None,
'ColumnD': (
'Int16',
-9, {
'len': [1, 2, (4-8)],
'val': [0, 1, (3-9)]
}
}
Returns:
(pandas.DataFrame): Result.
"""
This are the original messages:
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:11: WARNING: Block quote ends without a blank line; unexpected unindent.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:15: ERROR: Unexpected indentation.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:17: ERROR: Unexpected indentation.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:19: WARNING: Block quote ends without a blank line; unexpected unindent.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:20: WARNING: Block quote ends without a blank line; unexpected unindent.
reStructuredText is not Markdown, and indentation alone is not enough to demarcate the code block. reStructuredText calls this a literal block. Although the use of :: is one option, you might want to explicitly specify the language (overriding the default) with the use of the code-block directive.
Also I noticed that you have invalid syntax in your code block—a missing ) and extra spaces in your indentation—which could have caused those errors.
Try this.
def read_and_validate_csv(basename, specs_and_rules):
"""Read a CSV file with respect to specifications about format and
rules about valid values.
Hints: Do not use objects of type type (e.g. str instead of "str") when
specificing the column type.
.. code-block:: python
specs_and_rules = {
'TEMPLATES': {
'T1l': ('Int16', [-9, ' '])
},
'ColumnA': 'str',
'ColumnB': ('str', 'no answer'),
'ColumnC': None,
'ColumnD': (
'Int16',
-9, {
'len': [1, 2, (4-8)],
'val': [0, 1, (3-9)]
}
)
}
Returns:
(pandas.DataFrame): Result.
"""
Please find the GitLab repo for the terraform scripts which we are using.
enter link description here
Run in terraform plan gives the below error in an all-in-one.YAML file for the elastic search.
Error: Error in function call
on kubernetes.tf line 49, in locals:
49: resource_list = yamldecode(file("${path. module}/all-in-one.yaml")).items
|----------------
| path.module is "."
Call to function `"yamldecode"` failed: on line 458, column 1: unexpected extra
content after value.
enter image description here
As is describe in the fine manual:
Only one YAML document is permitted. If multiple documents are present in the given string then this function will return an error.
and one can trivially reproduce your error message:
content = yamldecode("---\nhello: world\n---\ntoo: bad\n")
on main.tf line 14, in resource "local_file" "example":
14: content = yamldecode("---\nhello: world\n---\ntoo: bad\n")
Call to function "yamldecode" failed: on line 2, column 1: unexpected extra
content after value.
I need to change the FreeMarker behavior about processing template&data in case some data are missing.
In missing case I need to let the template unchanged and continue in processing.
Example:
Template:
...
var1 = ${var1}
var2 = ${var2}
...
Data:
var1 = Hello
I need Result after processing:
...
var1 = Hello
var2 = ${var2}
...
My solution is:
class MyTemplateExceptionHandler implements TemplateExceptionHandler {
public void handleTemplateException(TemplateException te, Environment env, java.io.Writer out)
throws TemplateException {
try {
//2nd word on 2nd row is name of missing variable :(
String missingVariable = te.getMessageWithoutStackTop().split("\n")[1].split(" ")[1];
out.write("${" + missingVariable + "}");
} catch (IOException e) {
throw new TemplateException("Failed to print error message. Cause: " + e, env);
}
} }
and using:
...
Configuration cfg = new Configuration();
cfg.setTemplateExceptionHandler(new MyTemplateExceptionHandler());
...
My solution works, but I am not satisfied because of:
1) getting missing variable in handleTemplateException() is horrible. Does exist better way how to get a name of missing variable?
2) Even if I have my own TemplateExceptionHandler, FreeMarker writes enough information to output. Can I change it?
Example output for missing variable "environment_name":
22.10.2014 9:01:55 freemarker.log._JDK14LoggerFactory$JDK14Logger error
SEVERE: Error executing FreeMarker template
FreeMarker template error:
The following has evaluated to null or missing:
==> environment_name [in template "Osb-PrepareAndDeploy.ftl" at line 33, column 33]
Tip: If the failing expression is known to be legally null/missing, either specify a default value with myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthessis: (myOptionVar.foo)!myDefault, (myOptionVar.foo)??
The failing instruction (FTL stack trace):
----------
==> ${environment_name} [in template "Osb-PrepareAndDeploy.ftl" at line 33, column 31]
----------
Java stack trace (for programmers):
----------
freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...]
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:98)
at freemarker.core.EvalUtil.coerceModelToString(EvalUtil.java:382)
at freemarker.core.Expression.evalAndCoerceToString(Expression.java:115)
at freemarker.core.DollarVariable.accept(DollarVariable.java:76)
at freemarker.core.Environment.visit(Environment.java:265)
at freemarker.core.MixedContent.accept(MixedContent.java:93)
at freemarker.core.Environment.visit(Environment.java:265)
at freemarker.core.Environment.process(Environment.java:243)
at freemarker.template.Template.process(Template.java:277)
at net.homecredit.scm.jenkinsTool.countrySettings.Country.createJobs(Country.java:282)
at net.homecredit.scm.jenkinsTool.Start.main(Start.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
You can't prevent the exception from being logged. TemplateExceptionHandler doesn't mean to make non-error from an error, it just lets you decide what to do after the error has happened. Like, you want to print an error page or something. In extreme case, you may want to continue page rendering after printing some error indicator, but that's still an error that need to be fixed.
Maybe it would help if you tell why do you need this. Maybe the solution isn't TemplateExceptionHandler-s.
As of more intelligent exception processing, first, in the handler you should check if the exception is a InvalidReferenceException, becaue I guess you don't want to deal with others. Then, with getBlamedExpressionString() (requires FreeMarker 2.3.21) you could get part of what you want to print to the output. It's only part of that, because if ${x + 1} fails because x is undefined, it will only return "x", not "x + 1".
editable.ftl
[#ftl]
[#macro editable field ]
[#if field??]
[#if isEditable]
<h:inputext value="${field}" />
[#else]
${field?eval}
[/#if]
[/#if]
[/#macro]
what is the right way to import this to my templates. i tried with cfg.addAutoImport("me", "editable.ftl").
and following exception is thrown when i tried to process my main Template.
ERROR [freemarker.runtime] (http-localhost-127.0.0.1-8080-1) Error executing FreeMarker template: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> field [in template "editable" at line 5, column 30]
Tip: If the failing expression is known to be legally null/missing, either specify a default value with myOptionalVar!myDefault, or use [#if myOptionalVar??]when-present[#else]when-missing[/#if]. (These only cover the last step of the expression; to cover the whole expression, use parenthessis: (myOptionVar.foo)!myDefault, (myOptionVar.foo)??
The failing instruction:
==> ${field} [in template "editable" at line 5, column 28]
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:98) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.EvalUtil.coerceModelToString(EvalUtil.java:382) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Expression.evalAndCoerceToString(Expression.java:115) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.DollarVariable.accept(DollarVariable.java:76) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Environment.visit(Environment.java:265) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.MixedContent.accept(MixedContent.java:93) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Environment.visit(Environment.java:265) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Environment.include(Environment.java:1712) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Environment.importLib(Environment.java:1784) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Environment.importLib(Environment.java:1733) [freemarker-2.3.20.jar:2.3.20]
at freemarker.template.Configuration.doAutoImportsAndIncludes(Configuration.java:1105) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Configurable.doAutoImportsAndIncludes(Configurable.java:1271) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Configurable.doAutoImportsAndIncludes(Configurable.java:1271) [freemarker-2.3.20.jar:2.3.20]
at freemarker.core.Environment.process(Environment.java:242) [freemarker-2.3.20.jar:2.3.20]
The macros code should be executed when invoked but here seems they are invoked during import. Is it treated here really a macro? Please guide me what is wrong with this.
Macros aren't executed when importing (of course, unless you call them in the imported FTL itself outside a macro). This looks like an issue with the tag syntax selection. If FTL doesn't switch to [#...] tag syntax from the default <#...> tag syntax, then the only non-static-text parts will be the ${...}-s, and hence field won't be defined in your case. So the question is why it doesn't switch when you start the template with [#ftl]. Maybe you have an invisible BOM at the beginning of the file (Windows Notepad for example adds a BOM when editing UTF-8 files), or something else before the [#ftl].
BTW, setting tag_syntax to auto_detect prevents such surprises. It's not the default for backward compatibility...
I started using SST (selenium simple test) but ran into a problem when executing what seems to be a valid xpath expression '//div[div#data-type="folder-name"]'. SST fails with the following traceback:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/sst/cases.py", line 207, in run_test_script exec self.code in self.context
File "./sst-one.py", line 7, in <module> names = get_elements_by_xpath('//div[div#data-type="folder-name"]')
File "/usr/local/lib/python2.7/dist-packages/sst/actions.py", line 1344, in >get_elements_by_xpath_raise(msg)
File "/usr/local/lib/python2.7/dist-packages/sst/actions.py", line 118, in _raise raise AssertionError(msg)
AssertionError: Element not found: Message: u'The given selector //div[div#data->type="folder-name"] is either invalid or does not result in a WebElement. The following >error occurred:\nInvalidSelectorError: Unable to locate an element with the xpath >expression //div[div#data-type="folder-name"] because of the following >error:\n[Exception... "The expression is not a legal expression." code: "12" nsresult: >"0x805b0033 (SyntaxError)" location: "file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js> Line: 5916"]' ;
Stacktrace:
at FirefoxDriver.annotateInvalidSelectorError_ (file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js:8873)
at FirefoxDriver.prototype.findElementsInternal_ (file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js:8931)
at FirefoxDriver.prototype.findElements file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js:8935)
at DelayedCommand.prototype.executeInternal_/h file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/command_processor.js:10840)
at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/command_processor.js:10845)
at DelayedCommand.prototype.execute/< file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/command_processor.js:10787)
The SST code line is:
names = get_elements_by_xpath('//div[div#data-type="folder-name"]')
If I run the same xpath statement using selenium (from python), not SST, it is a valid expression and returns the matching elements if there were any.
The pure selenium code line (where br is the firefox webdriver object) is:
elems = br.find_elements_by_xpath("//div[div/#data-type='folder-name']")
I understand this is a bit in the weeds, so if anyone has some hints as to how to debug the .js code that selenium creates in the /tmp directory that would be a big help.
It's not a valid XPath expression. This
//div[div#data-type="folder-name"]
Should probably read
//div[div/#data-type="folder-name"]
which is indeed what you have in one of your examples.
If you are trying to get all possible div tags with data-type="folder-name",
//div[#data-type="folder-name"]
If only children of the first div with data-type="folder-name",
//div[div/#data-type="folder-name"]
You don't show any markup, up i assuming you have
<div><div data-type="folder-name"></div></div>
you need:
elems = br.find_elements_by_xpath("//div//div[#data-type='folder-name']")