Discussion:
[FreeMarker-user] Introducing freemarker-introspection
Wong, Christopher
2014-06-13 13:03:49 UTC
Permalink
I would like to introduce a new project, "freemarker-introspection". Meant to supplement the core Freemarker project, the purpose of this project is to provide a programmatic way to inspect and modify a Freemarker template. While Freemarker has an object representation of a parsed template, this representation is generally not externally accessible. The primary feature of freemarker-introspection is to expose the parsed Freemarker template as a DOM-like AST object that you can navigate in Java:

Element root = TemplateIntrospector.getRootNode(template);
for (Element child : root.getChildren()) {
for (Expr param : child.getParams()) {
...
}
}

You can also use a Visitor pattern to walk the template nodes. I implemented 2 classes that serve as consumers of this output.

VariableFinder scans the template and returns information on variables that the template will reference from the model:

Element root = TemplateIntrospector.getRootNode(template);
List<VariableInfo> vars = new VariableFinder(root).seek().getVariableInfo();

TemplateEditor lets you replace specific nodes (elements and expressions) with your own content:

Element root = TemplateIntrospector.getRootNode(template);
Element elem = ...; // find element node in tree to replace
Expr expr = ...; // find expression node in tree to replace
String newTemplateText = new TemplateEditor(oldTemplateText)
.replace(elem, "<#include 'something_else.ftl'>")
.replace(expr, "123")
.apply()
.getModifiedTemplate();

You can find freemarker-introspection at:

https://github.com/cwong15/freemarker-introspection

You will need Gradle to build it. It currently works with Freemarker versions 2.3.19 and 2.3.20. This project is still in its very early stages, so your feedback will be very much welcome.

Chris

________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Daniel Dekany
2014-06-14 11:09:45 UTC
Permalink
Do you plan to work on this against 2.3.21, or if something is missing
from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much easier
to keep freemarker-introspection in sync with it if it relies on
freemkarker.core-package-access rather than on reflection.
Post by Wong, Christopher
I would like to introduce a new project,
“freemarker-introspection”. Meant to supplement the core Freemarker
project, the purpose of this project is to provide a programmatic
way to inspect and modify a Freemarker template. While Freemarker
has an object representation of a parsed template, this
representation is generally not externally accessible. The primary
feature of freemarker-introspection is to expose the parsed
Element root = TemplateIntrospector.getRootNode(template);
for (Element child : root.getChildren()) {
for (Expr param : child.getParams()) {

}
}
You can also use a Visitor pattern to walk the template nodes. I
implemented 2 classes that serve as consumers of this output.
VariableFinder scans the template and returns information on
Element root = TemplateIntrospector.getRootNode(template);
List<VariableInfo> vars = new
VariableFinder(root).seek().getVariableInfo();
TemplateEditor lets you replace specific nodes (elements and
Element root = TemplateIntrospector.getRootNode(template);
Element elem = ...; // find element node in tree to replace
Expr expr = ...; // find expression node in tree to replace
String newTemplateText = new TemplateEditor(oldTemplateText)
.replace(elem, "<#include 'something_else.ftl'>")
.replace(expr, "123")
.apply()
.getModifiedTemplate();
https://github.com/cwong15/freemarker-introspection
You will need Gradle to build it. It currently works with
Freemarker versions 2.3.19 and 2.3.20. This project is still in its
very early stages, so your feedback will be very much welcome.
Chris
This e-mail and files transmitted with it are confidential, and are
intended solely for the use of the individual or entity to whom this
e-mail is addressed. If you are not the intended recipient, or the
employee or agent responsible to deliver it to the intended
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly
prohibited. If you are not one of the named recipient(s) or
otherwise have reason to believe that you received this message in
error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
--
Thanks,
Daniel Dekany
Wong, Christopher
2014-06-16 12:51:00 UTC
Permalink
I agree that the current reflection-based implementation is less maintainable than using the package-access methods you have available starting in 2.3.20. The current approach is only temporary, to allow support for an older version of Freemarker. I will replace it with your recommended approach. This list announcement is to solicit feedback on the public interfaces for freemarker-introspection, which should be independent of the underlying implementation.

Chris

-----Original Message-----
From: Daniel Dekany [mailto:***@freemail.hu]
Sent: Saturday, June 14, 2014 7:10 AM
To: FreeMarker-user
Subject: Re: [FreeMarker-user] Introducing freemarker-introspection

Do you plan to work on this against 2.3.21, or if something is missing from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much easier to keep freemarker-introspection in sync with it if it relies on freemkarker.core-package-access rather than on reflection.
Post by Wong, Christopher
I would like to introduce a new project, "freemarker-introspection".
Meant to supplement the core Freemarker project, the purpose of this
project is to provide a programmatic way to inspect and modify a
Freemarker template. While Freemarker has an object representation of
a parsed template, this representation is generally not externally
accessible. The primary feature of freemarker-introspection is to
expose the parsed Freemarker template as a DOM-like AST object that
Element root = TemplateIntrospector.getRootNode(template);
for (Element child : root.getChildren()) {
for (Expr param : child.getParams()) {
...
}
}
You can also use a Visitor pattern to walk the template nodes. I
implemented 2 classes that serve as consumers of this output.
VariableFinder scans the template and returns information on variables
Element root = TemplateIntrospector.getRootNode(template);
List<VariableInfo> vars = new
VariableFinder(root).seek().getVariableInfo();
TemplateEditor lets you replace specific nodes (elements and
Element root = TemplateIntrospector.getRootNode(template);
Element elem = ...; // find element node in tree to replace
Expr expr = ...; // find expression node in tree to replace
String newTemplateText = new TemplateEditor(oldTemplateText)
.replace(elem, "<#include 'something_else.ftl'>")
.replace(expr, "123")
.apply()
.getModifiedTemplate();
https://github.com/cwong15/freemarker-introspection
You will need Gradle to build it. It currently works with Freemarker
versions 2.3.19 and 2.3.20. This project is still in its very early
stages, so your feedback will be very much welcome.
Chris
This e-mail and files transmitted with it are confidential, and are
intended solely for the use of the individual or entity to whom this
e-mail is addressed. If you are not the intended recipient, or the
employee or agent responsible to deliver it to the intended recipient,
you are hereby notified that any dissemination, distribution or
copying of this communication is strictly prohibited. If you are not
one of the named recipient(s) or otherwise have reason to believe that
you received this message in error, please immediately notify sender
by e-mail, and destroy the original message. Thank You.
--
Thanks,
Daniel Dekany


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
FreeMarker-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freemarker-user

This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Jonathan Revusky
2014-06-17 12:46:34 UTC
Permalink
You know, there is a 2.4 codebase which I spent a huge amount of time
refactoring in order to make this kind of project technically feasible.
THere is all kinds of machinery in there for tree-walking design patterns
where you walk the AST and modify things. IN fact, I recall years ago,
Attila telling me that some people in the company he was working at
(Adeptra?) were already using that to great effect.

I still have no real understanding of why we never moved to that codebase.
I was sure it was a very big mistake, and this just reinforces that belief,
but, at the time, I was losing energy on the whole thing and didn't push on
the whole thing.

By the way, I guess the 2.4 is still in the sourceforge SVN repository. Or
is it?
Post by Wong, Christopher
I agree that the current reflection-based implementation is less
maintainable than using the package-access methods you have available
starting in 2.3.20. The current approach is only temporary, to allow
support for an older version of Freemarker. I will replace it with your
recommended approach. This list announcement is to solicit feedback on the
public interfaces for freemarker-introspection, which should be independent
of the underlying implementation.
Chris
-----Original Message-----
Sent: Saturday, June 14, 2014 7:10 AM
To: FreeMarker-user
Subject: Re: [FreeMarker-user] Introducing freemarker-introspection
Do you plan to work on this against 2.3.21, or if something is missing
from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much easier to
keep freemarker-introspection in sync with it if it relies on
freemkarker.core-package-access rather than on reflection.
Post by Wong, Christopher
I would like to introduce a new project, "freemarker-introspection".
Meant to supplement the core Freemarker project, the purpose of this
project is to provide a programmatic way to inspect and modify a
Freemarker template. While Freemarker has an object representation of
a parsed template, this representation is generally not externally
accessible. The primary feature of freemarker-introspection is to
expose the parsed Freemarker template as a DOM-like AST object that
Element root = TemplateIntrospector.getRootNode(template);
for (Element child : root.getChildren()) {
for (Expr param : child.getParams()) {
...
}
}
You can also use a Visitor pattern to walk the template nodes. I
implemented 2 classes that serve as consumers of this output.
VariableFinder scans the template and returns information on variables
Element root = TemplateIntrospector.getRootNode(template);
List<VariableInfo> vars = new
VariableFinder(root).seek().getVariableInfo();
TemplateEditor lets you replace specific nodes (elements and
Element root = TemplateIntrospector.getRootNode(template);
Element elem = ...; // find element node in tree to replace
Expr expr = ...; // find expression node in tree to replace
String newTemplateText = new TemplateEditor(oldTemplateText)
.replace(elem, "<#include 'something_else.ftl'>")
.replace(expr, "123")
.apply()
.getModifiedTemplate();
https://github.com/cwong15/freemarker-introspection
You will need Gradle to build it. It currently works with Freemarker
versions 2.3.19 and 2.3.20. This project is still in its very early
stages, so your feedback will be very much welcome.
Chris
This e-mail and files transmitted with it are confidential, and are
intended solely for the use of the individual or entity to whom this
e-mail is addressed. If you are not the intended recipient, or the
employee or agent responsible to deliver it to the intended recipient,
you are hereby notified that any dissemination, distribution or
copying of this communication is strictly prohibited. If you are not
one of the named recipient(s) or otherwise have reason to believe that
you received this message in error, please immediately notify sender
by e-mail, and destroy the original message. Thank You.
--
Thanks,
Daniel Dekany
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
https://lists.sourceforge.net/lists/listinfo/freemarker-user
This e-mail and files transmitted with it are confidential, and are
intended solely for the use of the individual or entity to whom this e-mail
is addressed. If you are not the intended recipient, or the employee or
agent responsible to deliver it to the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this
communication is strictly prohibited. If you are not one of the named
recipient(s) or otherwise have reason to believe that you received this
message in error, please immediately notify sender by e-mail, and destroy
the original message. Thank You.
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
https://lists.sourceforge.net/lists/listinfo/freemarker-user
Daniel Dekany
2014-06-17 19:13:25 UTC
Permalink
Post by Jonathan Revusky
You know, there is a 2.4 codebase which I spent a huge amount of
time refactoring in order to make this kind of project technically
feasible. THere is all kinds of machinery in there for tree-walking
design patterns where you walk the AST and modify things. IN fact, I
recall years ago, Attila telling me that some people in the company
he was working at (Adeptra?) were already using that to great effect.
I still have no real understanding of why we never moved to that
codebase. I was sure it was a very big mistake,
The "mistake" was, simply, not investing a lot of time (including busy
work...) into FM development in the recent several years. Whether it
goes into that cancelled 2.4 (later 3.0) branch or it goes into yet
another branch is a question of details, compared to that. But I can
only call this a mistake in quotation marks because nobody pays you or
Attila or me to do it. And, 2.3.x had to be maintained, or else you
can just delete the whole project. The DLTK fiasco didn't helped 3.0
either. Anyway, I'm still trying to push this thing into something
that can be called 2.4, even if it meant to be different that the
cancelled 2.4/3.0 was.

But let's not discus this here, if there's anything to discuss.
Post by Jonathan Revusky
and this just reinforces that belief, but, at the time, I was losing
energy on the whole thing and didn't push on the whole thing. By the
way, I guess the 2.4 is still in the sourceforge SVN repository. Or
is it?
It has to be.
Post by Jonathan Revusky
I agree that the current reflection-based implementation is less
maintainable than using the package-access methods you have
available starting in 2.3.20. The current approach is only
temporary, to allow support for an older version of Freemarker. I
will replace it with your recommended approach. This list
announcement is to solicit feedback on the public interfaces for
freemarker-introspection, which should be independent of the underlying implementation.
Chris
-----Original Message-----
Sent: Saturday, June 14, 2014 7:10 AM
To: FreeMarker-user
Subject: Re: [FreeMarker-user] Introducing freemarker-introspection
Do you plan to work on this against 2.3.21, or if something is
missing from the freemarker.core-package-visible API, then against the trunk?
If I refactor something inside freemarker.core later, it's much
easier to keep freemarker-introspection in sync with it if it relies
on freemkarker.core-package-access rather than on reflection.
Post by Wong, Christopher
I would like to introduce a new project, "freemarker-introspection".
Meant to supplement the core Freemarker project, the purpose of this
project is to provide a programmatic way to inspect and modify a
Freemarker template. While Freemarker has an object representation of
a parsed template, this representation is generally not externally
accessible. The primary feature of freemarker-introspection is to
expose the parsed Freemarker template as a DOM-like AST object that
Element root = TemplateIntrospector.getRootNode(template);
for (Element child : root.getChildren()) {
for (Expr param : child.getParams()) {
...
}
}
You can also use a Visitor pattern to walk the template nodes. I
implemented 2 classes that serve as consumers of this output.
VariableFinder scans the template and returns information on variables
Element root = TemplateIntrospector.getRootNode(template);
List<VariableInfo> vars = new
VariableFinder(root).seek().getVariableInfo();
TemplateEditor lets you replace specific nodes (elements and
Element root = TemplateIntrospector.getRootNode(template);
Element elem = ...; // find element node in tree to replace
Expr expr = ...; // find expression node in tree to replace
String newTemplateText = new TemplateEditor(oldTemplateText)
.replace(elem, "<#include 'something_else.ftl'>")
.replace(expr, "123")
.apply()
.getModifiedTemplate();
https://github.com/cwong15/freemarker-introspection
You will need Gradle to build it. It currently works with Freemarker
versions 2.3.19 and 2.3.20. This project is still in its very early
stages, so your feedback will be very much welcome.
Chris
This e-mail and files transmitted with it are confidential, and are
intended solely for the use of the individual or entity to whom this
e-mail is addressed. If you are not the intended recipient, or the
employee or agent responsible to deliver it to the intended recipient,
you are hereby notified that any dissemination, distribution or
copying of this communication is strictly prohibited. If you are not
one of the named recipient(s) or otherwise have reason to believe that
you received this message in error, please immediately notify sender
by e-mail, and destroy the original message. Thank You.
--
Thanks,
Daniel Dekany
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
https://lists.sourceforge.net/lists/listinfo/freemarker-user
This e-mail and files transmitted with it are confidential, and are
intended solely for the use of the individual or entity to whom this
e-mail is addressed. If you are not the intended recipient, or the
employee or agent responsible to deliver it to the intended
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly
prohibited. If you are not one of the named recipient(s) or
otherwise have reason to believe that you received this message in
error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
FreeMarker-user mailing list
https://lists.sourceforge.net/lists/listinfo/freemarker-user
--
Thanks,
Daniel Dekany
Loading...