Recently I coded an Obfuscator with ASM in Java and wanted to rename classes, methods, and fields. But the problem is, that the code doesn't work it should too, and I have no clue how to fix that. The problem is, that if I obfuscate a jar every method in the class gets renamed, but sometimes (not every time) a bit of code is not getting renamed, so the jar can't be executed. E.g.
public abstract class ColorThread implements Runnable
{
#Getter
private final String name;
#Getter
private Thread thread;
public ColorThread(final String name) {
this.name = name;
Runtime.getRuntime().addShutdownHook(new Thread(this::close));
}
#Override
public void run() {
throw new NotOverriddenException("The Thread \"" + getName() + "\" is not overwritten.");
}
/**
* This method interrupts the running thread.
*/
public void close() {
this.getThread().interrupt();
}
public void start() { //<- Method gets renamed e.g "⢍⢖⣕⠟⡨⠣"
this.thread = new Thread(this, this.getName());
thread.start();
}
}
So this class got obfuscated but later in other code which calls:
final ConnectThread connectThread = new ConnectThread();
connectThread.start(); // <- this line
the line with connectThread.start(); isn't renamed to "connectThread.⢍⢖⣕⠟⡨⠣();". If I use another class which extends ColorThread e.g. ReceiveThread, the start method gets renamed in this bit of code.
I struggled every time with this problem if I made an Obfuscator and because of it I ended the project. But now I want to ask here if someone can help me. Sorry for this long post, but I wanted to give everything needed to see the problem.
The Project is running on Java 1.8.0_161 with ASM-All as a dependency.
To read a jar i use this method. It will store all classes in an ArrayList:
try (final JarFile jarFile = new JarFile(inputFile)) {
final Enumeration<JarEntry> jarEntryEnumeration = jarFile.entries();
while (jarEntryEnumeration.hasMoreElements()) {
final JarEntry jarEntry = jarEntryEnumeration.nextElement();
if (jarEntry.isDirectory())
continue;
final byte[] bytes = this.readInputStream(jarFile.getInputStream(jarEntry));
if (jarEntry.getName().endsWith(".class")) {
if (jarEntry.getName().endsWith("module-info.class"))
continue;
final ClassNode classNode = new ClassNode();
// new ClassReader(bytes).accept(classNode, ClassReader.EXPAND_FRAMES | ClassReader.SKIP_DEBUG);
new ClassReader(bytes).accept(classNode, ClassReader.EXPAND_FRAMES);
this.classes.add(classNode);
} else {
if (jarEntry.getName().contains("MANIFEST.MF"))
continue;
this.files.put(jarEntry.getName(), bytes);
}
}
this.manifest = jarFile.getManifest();
} catch (final Exception ex) {
ex.printStackTrace();
}
After this i use my transformation system to rename the methods:
#Override
public void transform(final ArrayList<ClassNode> classes, final HashMap<String, byte[]> files) {
final String mainClass = this.getJarResources().getManifest().getMainAttributes().getValue("Main-Class").replace(".", "/");
final HashMap<String, String> methodNames = new HashMap<>();
for (final ClassNode classNode : classes) {
for (final Object methodObj : classNode.methods) {
if (!(methodObj instanceof MethodNode))
continue;
final MethodNode methodNode = (MethodNode) methodObj;
if (methodNode.name.equals("<init>"))
continue;
if (methodNode.name.equals(mainClass) || methodNode.name.equals("main"))
continue;
methodNames.put(classNode.name + "." + methodNode.name + methodNode.desc, this.generateString(6));
}
}
this.remapClasses(classes, methodNames);
}
The remap method looks like this:
public void remapClasses(final ArrayList<ClassNode> classes, final HashMap<String, String> remappedNames) {
final SimpleRemapper simpleRemapper = new SimpleRemapper(remappedNames);
for (int index = 0; index < classes.size(); index++) {
final ClassNode realNode = classes.get(index);
final ClassNode copyNode = new ClassNode();
final ClassRemapper classRemapper = new ClassRemapper(copyNode, simpleRemapper);
realNode.accept(classRemapper);
classes.set(index, copyNode);
}
}
At the end i write the file:
public void writeFile() {
try (final JarOutputStream jarOutputStream = new JarOutputStream(new FileOutputStream(this.outputFile), this.manifest)) {
for (final ClassNode classNode : this.classes) {
final ClassWriter writer = new ClassWriter(ClassWriter.COMPUTE_MAXS);
classNode.accept(writer);
jarOutputStream.putNextEntry(new JarEntry(classNode.name + ".class"));
jarOutputStream.write(writer.toByteArray());
jarOutputStream.closeEntry();
}
for (final Map.Entry<String, byte[]> file : this.files.entrySet()) {
final String filePath = file.getKey();
if(filePath.endsWith(".kotlin_module") || filePath.contains("maven") || filePath.contains("3rd-party-licenses"))
continue;
jarOutputStream.putNextEntry(new JarEntry(filePath));
jarOutputStream.write(file.getValue());
jarOutputStream.closeEntry();
}
} catch (final Exception ex) {
ex.printStackTrace();
}
}
Related
I have started working on a Freemarker Debugger using breakpoints etc. The supplied framework is based on java RMI. So far I get it to suspend at one breakpoint but then ... nothing.
Is there a very basic example setup for the serverpart and the client part other then the debug/imp classes supplied with the sources. That would be of great help.
this is my server class:
class DebuggerServer {
private final int port;
private final String templateName1;
private final Environment templateEnv;
private boolean stop = false;
public DebuggerServer(String templateName) throws IOException {
System.setProperty("freemarker.debug.password", "hello");
port = SecurityUtilities.getSystemProperty("freemarker.debug.port", Debugger.DEFAULT_PORT).intValue();
System.setProperty("freemarker.debug.password", "hello");
Configuration cfg = new Configuration();
// Some other recommended settings:
cfg.setIncompatibleImprovements(new Version(2, 3, 20));
cfg.setDefaultEncoding("UTF-8");
cfg.setLocale(Locale.US);
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
Template template = cfg.getTemplate(templateName);
templateName1 = template.getName();
System.out.println("Debugging " + templateName1);
Map<String, Object> root = new HashMap();
Writer consoleWriter = new OutputStreamWriter(System.out);
templateEnv = new Environment(template, null, consoleWriter);
DebuggerService.registerTemplate(template);
}
public void start() {
new Thread(new Runnable() {
#Override
public void run() {
startInternal();
}
}, "FreeMarker Debugger Server Acceptor").start();
}
private void startInternal() {
boolean handled = false;
while (!stop) {
List breakPoints = DebuggerService.getBreakpoints(templateName1);
for (int i = 0; i < breakPoints.size(); i++) {
try {
Breakpoint bp = (Breakpoint) breakPoints.get(i);
handled = DebuggerService.suspendEnvironment(templateEnv, templateName1, bp.getLine());
} catch (RemoteException e) {
System.err.println(e.getMessage());
}
}
}
}
public void stop() {
this.stop = true;
}
}
This is the client class:
class DebuggerClientHandler {
private final Debugger client;
private boolean stop = false;
public DebuggerClientHandler(String templateName) throws IOException {
// System.setProperty("freemarker.debug.password", "hello");
// System.setProperty("java.rmi.server.hostname", "192.168.2.160");
client = DebuggerClient.getDebugger(InetAddress.getByName("localhost"), Debugger.DEFAULT_PORT, "hello");
client.addDebuggerListener(environmentSuspendedEvent -> {
System.out.println("Break " + environmentSuspendedEvent.getName() + " at line " + environmentSuspendedEvent.getLine());
// environmentSuspendedEvent.getEnvironment().resume();
});
}
public void start() {
new Thread(new Runnable() {
#Override
public void run() {
startInternal();
}
}, "FreeMarker Debugger Server").start();
}
private void startInternal() {
while (!stop) {
}
}
public void stop() {
this.stop = true;
}
public void addBreakPoint(String s, int i) throws RemoteException {
Breakpoint bp = new Breakpoint(s, i);
List breakpoints = client.getBreakpoints();
client.addBreakpoint(bp);
}
}
Liferay IDE (https://github.com/liferay/liferay-ide) has FreeMarker template debug support (https://issues.liferay.com/browse/IDE-976), so somehow they managed to use it. I have never seen it in action though. Other than that, I'm not aware of anything that uses the debug API.
I have few MongoTemplate and Repos and i need to call them using #Autowire in my runnable class that is being executed by exceutor class using multi threading, now the problem is that when i run the application my AutoWire for mongoTempelate and Repos returns null pointer exception.
Executor class:
#Component
public class MessageConsumer implements ConsumerSeekAware {
#Autowired
AlarmDataRepository alarmDataRepository;
int assignableCores = ((Runtime.getRuntime().availableProcessors()));
ExecutorService executor = Executors.newFixedThreadPool(
assignableCores > 1 ? assignableCores : 1
);
int counter = 0;
List<String> uniqueRecords = new ArrayList<String>();
#KafkaListener(topics = "teltonikaTest", groupId = "xyz")
public void processMessages(#Payload List<String> payload, #Header(KafkaHeaders.RECEIVED_PARTITION_ID) List<Integer> partitions, #Header(KafkaHeaders.OFFSET) List<Long> offsets) throws UnsupportedEncodingException, DecodeException {
System.out.println("assignable resources are: " + assignableCores);
log.info("Batch Size is: {}", payload.size());
if(counter==0){
log.info("Teletonica Packets Received!");
}
for (int i = 0; i < payload.size(); i++) {
log.info("processing message='{}' with partition off-set='{}'", payload.get(i), partitions.get(i) + " _" + offsets.get(i));
}
uniqueRecords = payload.stream().distinct().collect(Collectors.toList());
Runnable worker = new TeltonikaWorkerThread(uniqueRecords);
executor.execute(worker);
counter++;
}
}
public class TeltonikaWorkerThread implements Runnable{
List<String> records;
List<CurrentDevice> currentDevices = new ArrayList<>();
#Autowired
CurrentDeviceRepository currentDeviceRepository;
#Autowired
MongoTemplate mongoTemplate;
public TeltonikaWorkerThread(List<String> records) {
this.records = records;
}
public void run() {
try {
processMessage();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (DecodeException e) {
e.printStackTrace();
}
}
public void processMessage() throws UnsupportedEncodingException,DecodeException {
for(Object record : records){
if(record!="0"){
try{
int IMEILength = record.toString().indexOf("FF");
String IMEI = record.toString().substring(0,IMEILength);
}
catch (Exception e){
e.printStackTrace();
}
}
}
}
}
If I understand correctly, your problem is about multiple beans and Spring doesn't know which one should be injected. There are several options here.
For example, you can use #Qualifier annotation based on the bean name or #Primary annotation.
If your problem is something else, please add an example to your question.
I'd like to write end-to-end tests to validate two boot apps work well together with various profiles.
What already works:
create a third maven module (e2e) for end-to-end tests, in addition to the two tested apps (authorization-server and resource-server)
write tests using TestResTemplate
Test work fine if I start authorization-server and resource-server manually.
What I now want to do is automate the tested boot apps startup and shutdown with the right profiles for each test.
I tried:
adding maven dependencies to tested apps in e2e module
using SpringApplication in new threads for each app to start
But I face miss-configuration issues as all resources and dependencies end in the same shared classpath...
Is there a way to sort this out?
I'm also considering starting two separate java -jar ... processes, but then, how to ensure tested apps fat-jars are built before 2e2 unit-tests run?
Current app start/shutdown code sample which fails as soon as I had maven dependency to second app to start:
private Service startAuthorizationServer(boolean isJwtActive) throws InterruptedException {
return new Service(
AuthorizationServer.class,
isJwtActive ? new String[]{ "jwt" } : new String[]{} );
}
private static final class Service {
private ConfigurableApplicationContext context;
private final Thread thread;
public Service(Class<?> appClass, String... profiles) throws InterruptedException {
thread = new Thread(() -> {
SpringApplication app = new SpringApplicationBuilder(appClass).profiles(profiles).build();
context = app.run();
});
thread.setDaemon(false);
thread.start();
while (context == null || !context.isRunning()) {
Thread.sleep(1000);
};
}
#PreDestroy
public void stop() {
if (context != null) {
SpringApplication.exit(context);
}
if (thread != null) {
thread.interrupt();
}
}
}
I think your case, running the two applications via a docker compose can be a good idea.
This article shows how you can set up some integration tests using a docker compose image: https://blog.codecentric.de/en/2017/03/writing-integration-tests-docker-compose-junit/
Also, take a look at this post from Martin Fowler: https://martinfowler.com/articles/microservice-testing/
I got things working with second solution:
end-to-end tests projects has no other maven dependency than what is required to run spring-tests with TestRestClient
test config initialises environment, running mvn packageon required modules in separate processes
test cases run (re)start apps with chosen profiles in separate java -jar ... processes
Here is the helper class I wrote for this (taken from there):
class ActuatorApp {
private final int port;
private final String actuatorEndpoint;
private final File jarFile;
private final TestRestTemplate actuatorClient;
private Process process;
private ActuatorApp(File jarFile, int port, TestRestTemplate actuatorClient) {
this.port = port;
this.actuatorEndpoint = getBaseUri() + "actuator/";
this.actuatorClient = actuatorClient;
this.jarFile = jarFile;
Assert.isTrue(jarFile.exists(), jarFile.getAbsolutePath() + " does not exist");
}
public void start(List<String> profiles, List<String> additionalArgs) throws InterruptedException, IOException {
if (isUp()) {
stop();
}
this.process = Runtime.getRuntime().exec(appStartCmd(jarFile, profiles, additionalArgs));
Executors.newSingleThreadExecutor().submit(new ProcessStdOutPrinter(process));
for (int i = 0; i < 10 && !isUp(); ++i) {
Thread.sleep(5000);
}
}
public void start(String... profiles) throws InterruptedException, IOException {
this.start(Arrays.asList(profiles), List.of());
}
public void stop() throws InterruptedException {
if (isUp()) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON_UTF8);
headers.setAccept(List.of(MediaType.APPLICATION_JSON_UTF8));
actuatorClient.postForEntity(actuatorEndpoint + "shutdown", new HttpEntity<>(headers), Object.class);
Thread.sleep(5000);
}
if (process != null) {
process.destroy();
}
}
private String[] appStartCmd(File jarFile, List<String> profiles, List<String> additionalArgs) {
final List<String> cmd = new ArrayList<>(
List.of(
"java",
"-jar",
jarFile.getAbsolutePath(),
"--server.port=" + port,
"--management.endpoint.heath.enabled=true",
"--management.endpoint.shutdown.enabled=true",
"--management.endpoints.web.exposure.include=*",
"--management.endpoints.web.base-path=/actuator"));
if (profiles.size() > 0) {
cmd.add("--spring.profiles.active=" + profiles.stream().collect(Collectors.joining(",")));
}
if (additionalArgs != null) {
cmd.addAll(additionalArgs);
}
return cmd.toArray(new String[0]);
}
private boolean isUp() {
try {
final ResponseEntity<HealthResponse> response =
actuatorClient.getForEntity(actuatorEndpoint + "health", HealthResponse.class);
return response.getStatusCode().is2xxSuccessful() && response.getBody().getStatus().equals("UP");
} catch (ResourceAccessException e) {
return false;
}
}
public static Builder builder(String moduleName, String moduleVersion) {
return new Builder(moduleName, moduleVersion);
}
/**
* Configure and build a spring-boot app
*
* #author Ch4mp
*
*/
public static class Builder {
private String moduleParentDirectory = "..";
private final String moduleName;
private final String moduleVersion;
private int port = SocketUtils.findAvailableTcpPort(8080);
private String actuatorClientId = "actuator";
private String actuatorClientSecret = "secret";
public Builder(String moduleName, String moduleVersion) {
this.moduleName = moduleName;
this.moduleVersion = moduleVersion;
}
public Builder moduleParentDirectory(String moduleParentDirectory) {
this.moduleParentDirectory = moduleParentDirectory;
return this;
}
public Builder port(int port) {
this.port = port;
return this;
}
public Builder actuatorClientId(String actuatorClientId) {
this.actuatorClientId = actuatorClientId;
return this;
}
public Builder actuatorClientSecret(String actuatorClientSecret) {
this.actuatorClientSecret = actuatorClientSecret;
return this;
}
/**
* Ensures the app module is found and packaged
* #return app ready to be started
* #throws IOException if module packaging throws one
* #throws InterruptedException if module packaging throws one
*/
public ActuatorApp build() throws IOException, InterruptedException {
final File moduleDir = new File(moduleParentDirectory, moduleName);
packageModule(moduleDir);
final File jarFile = new File(new File(moduleDir, "target"), moduleName + "-" + moduleVersion + ".jar");
return new ActuatorApp(jarFile, port, new TestRestTemplate(actuatorClientId, actuatorClientSecret));
}
private void packageModule(File moduleDir) throws IOException, InterruptedException {
Assert.isTrue(moduleDir.exists(), "could not find module. " + moduleDir + " does not exist.");
String[] cmd = new File(moduleDir, "pom.xml").exists() ?
new String[] { "mvn", "-DskipTests=true", "package" } :
new String[] { "./gradlew", "bootJar" };
Process mvnProcess = new ProcessBuilder().directory(moduleDir).command(cmd).start();
Executors.newSingleThreadExecutor().submit(new ProcessStdOutPrinter(mvnProcess));
Assert.isTrue(mvnProcess.waitFor() == 0, "module packaging exited with error status.");
}
}
private static class ProcessStdOutPrinter implements Runnable {
private InputStream inputStream;
public ProcessStdOutPrinter(Process process) {
this.inputStream = process.getInputStream();
}
#Override
public void run() {
new BufferedReader(new InputStreamReader(inputStream)).lines().forEach(System.out::println);
}
}
public String getBaseUri() {
return "https://localhost:" + port;
}
}
I am newbee to Spring integration. i am trying to implement customer sftp filter to list the files in SFTP server. I am getting "The blank final field seen may not have been initialized" at the constructor.Can you please suggest me to get list of file names from sftp server.
I dont have any idea what went wrong in my code.
Thanks in Advance
java code
public class SFTPFileFilter extends SftpSimplePatternFileListFilter {
public SFTPFileFilter(String pattern) {
super(pattern);
// TODO Auto-generated constructor stub
}
final static Logger logger = LoggerFactory.getLogger(SFTPFileFilter.class);
private final Queue<File> seen;
private final Set<File> seenSet = new HashSet<File>();
private final Object monitor = new Object();
public static int fileCount = 0;
#Autowired
private SourcePollingChannelAdapter sftpInbondAdapter;
public List<File> filterFiles(File[] files)
{
List<File> accepted = new ArrayList<File>();
for (File file : files) {
System.out.println(file.getName());
accepted.add(file);
}
return accepted;
}
public boolean accept(File file) {
synchronized (this.monitor) {
if (this.seenSet.contains(file)) {
logger.info(file.getName()+" is already copied earlier");
return false;
}
if (this.seen != null) {
if (!this.seen.offer(file)) {
File removed = this.seen.poll();
this.seenSet.remove(removed);
this.seen.add(file);
}
}
this.seenSet.add(file);
return true;
}
}
}
private final Queue<File> seen;
You are not initializing that field in a constructor.
You can't extend it like that; simply override the method like this...
public List<File> filterFiles(File[] files) {
for (File file : files) {
System.out.println("received:" + file.getName());
}
List<File> filtered = super.filterFiles(files);
for (File file : flteredFiles) {
System.out.println("after filter:" + file.getName());
}
return filteredFiles;
}
I am using Freemarker and DCEVM+HotSwapManager agent. This basically allows me to hotswap classes even when adding/removing methods.
Everything works like charm until Freemarker uses hotswapped class as model. It's throwing freemarker.ext.beans.InvalidPropertyException: No such bean property on me even though reflection shows that the method is there (checked during debug session).
I am using
final Method clearInfoMethod = beanWrapper.getClass().getDeclaredMethod("removeIntrospectionInfo", Class.class);
clearInfoMethod.setAccessible(true);
clearInfoMethod.invoke(clazz);
to clear the cache, but it does not work. I even tried to obtain classCache member field and clear it using reflection but it does not work too.
What am I doing wrong?
I just need to force freemarker to throw away any introspection on model class/classes he has already obtained.
Is there any way?
UPDATE
Example code
Application.java
// Application.java
public class Application
{
public static final String TEMPLATE_PATH = "TemplatePath";
public static final String DEFAULT_TEMPLATE_PATH = "./";
private static Application INSTANCE;
private Configuration freemarkerConfiguration;
private BeansWrapper beanWrapper;
public static void main(String[] args)
{
final Application application = new Application();
INSTANCE = application;
try
{
application.run(args);
}
catch (InterruptedException e)
{
System.out.println("Exiting");
}
catch (IOException e)
{
System.out.println("IO Error");
e.printStackTrace();
}
}
public Configuration getFreemarkerConfiguration()
{
return freemarkerConfiguration;
}
public static Application getInstance()
{
return INSTANCE;
}
private void run(String[] args) throws InterruptedException, IOException
{
final String templatePath = System.getProperty(TEMPLATE_PATH) != null
? System.getProperty(TEMPLATE_PATH)
: DEFAULT_TEMPLATE_PATH;
final Configuration configuration = new Configuration();
freemarkerConfiguration = configuration;
beanWrapper = new BeansWrapper();
beanWrapper.setUseCache(false);
configuration.setObjectWrapper(beanWrapper);
try
{
final File templateDir = new File(templatePath);
configuration.setTemplateLoader(new FileTemplateLoader(templateDir));
}
catch (IOException e)
{
throw new RuntimeException(e);
}
final RunnerImpl runner = new RunnerImpl();
try
{
runner.run(args);
}
catch (RuntimeException e)
{
e.printStackTrace();
}
}
public BeansWrapper getBeanWrapper()
{
return beanWrapper;
}
}
RunnerImpl.java
// RunnerImpl.java
public class RunnerImpl implements Runner
{
#Override
public void run(String[] args) throws InterruptedException
{
long counter = 0;
while(true)
{
++counter;
System.out.printf("Run %d\n", counter);
// Application.getInstance().getFreemarkerConfiguration().setObjectWrapper(new BeansWrapper());
Application.getInstance().getBeanWrapper().clearClassIntrospecitonCache();
final Worker worker = new Worker();
worker.doWork();
Thread.sleep(1000);
}
}
Worker.java
// Worker.java
public class Worker
{
void doWork()
{
final Application application = Application.getInstance();
final Configuration freemarkerConfiguration = application.getFreemarkerConfiguration();
try
{
final Template template = freemarkerConfiguration.getTemplate("test.ftl");
final Model model = new Model();
final PrintWriter printWriter = new PrintWriter(System.out);
printObjectInto(model);
System.out.println("-----TEMPLATE MACRO PROCESSING-----");
template.process(model, printWriter);
System.out.println();
System.out.println("-----END OF PROCESSING------");
System.out.println();
}
catch (IOException e)
{
e.printStackTrace();
}
catch (TemplateException e)
{
e.printStackTrace();
}
}
private void printObjectInto(Object o)
{
final Class<?> aClass = o.getClass();
final Method[] methods = aClass.getDeclaredMethods();
for (final Method method : methods)
{
System.out.println(String.format("Method name: %s, public: %s", method.getName(), Modifier.isPublic(method.getModifiers())));
}
}
}
Model.java
// Model.java
public class Model
{
public String getMessage()
{
return "Hello";
}
public String getAnotherMessage()
{
return "Hello World!";
}
}
This example does not work at all. Even changing BeansWrapper during runtime won't have any effect.
BeansWrapper (and DefaultObjectWrapper's, etc.) introspection cache relies on java.beans.Introspector.getBeanInfo(aClass), not on reflection. (That's because it treats objects as JavaBeans.) java.beans.Introspector has its own internal cache, so it can return stale information, and in that case BeansWrapper will just recreate its own class introspection data based on that stale information. As of java.beans.Introspector's caching, it's in fact correct, as it builds on the assumption that classes in Java are immutable. If something breaks that basic rule, it should ensure that java.beans.Introspector's cache is cleared (and many other caches...), or else it's not just FreeMarker that will break. At JRebel for example they made a lot of effort to clear all kind of caches. I guess DCEVM doesn't have the resources for that. So then, it seems you have to call Introspector.flushCaches() yourself.
Update: For a while (Java 7, maybe 6) java.beans.Introspector has one cache per thread group, so you have call flushCaches() from all thread groups. And this all is actually implementation detail that, in principle, can change any time. And sadly, the JavaDoc of Introspector.flushCaches() doesn't warn you...