Update zum 10 April vom  4DSummit 2016

10. Apr: Mitchell Stiller auf 4D iNug Technical

Here is some of what I picked up at the Summit. I will not expand too much on what others have already mentioned. I have also included some tips that many of you might already know but somehow I had never encountered or learned before  or forgotten.  Ignore if they are too obvious.  Hopefully I got everything right, but chime in if I missed something.

4D The Business

4D continues to grow its business.  Sales were up about 10% last year.  Geographically a very diverse sales base.  With most revenues coming from recurrent sales.  All good things.  4D moved its San Jose and Paris offices in the last year.

4D Application Server

64 bit Developer Mac to be released soon.
Pre-emptive multi-threading in upcoming R-Releases as posted will require changes to coding behavior.
64-bit big advantage to everyone on Mac.  Multi-threading more advantageous for calculation and complex db’s.  Less so for data access.
Worker commands will allow inter-process communication without the need for inter-process variables.

Object Fields / Associated commands

QUERY BY ATTRIBUTE is a big deal.  Allows to create NoSQL style db’s or tables.  Never fully saw the advantage of this till Olivier’s demo. Often easier to set up than a related tables. i.e. all data in one table.  Can replace the functionality of the old sub-table.  Can use less space and be faster than related tables. e.g. I have to import external data on all medications produced in the country.  For each generic there are many formats (suspension, tablet etc) and several manufacturers and other attributes.  Structure would require a main table and about 6 related tables.  Can all be collapsed into one table with several object fields.
THE important thing is that if  you are storing “records” / arrays n the object field it is very important whether the data is stored as rows or columns.  Search  coding / results / speed will differ. You will be limited on what you can search for. If you are storing simple key value pairs in an object field you have no concerns.
You CANNOT do an ORDER BY on an object field at the present time.
Object fields can be indexed.  The only option is automatic.

If you really want to leverage 4D, learn to love objects.  Olivier’s presentation on mapping with Leaflet clearly demonstrated this. (There is a tech note by Thomas Maul on the same subject.  Olivier did a few other things in his.)  Likewise for graphing, select you data in 4D, generate your objects and export the object and send them to a javascript graphing package of your choice.

Objects allow you to write more generic robust code.  JPR did the typical JPR demo. Impressive, funny, brilliant… requiring a substantial time to de-code his demo. But in combination with the evolution to multi-threading, we will need to start to adapt our coding style to take advantage of this.
Related to this, is that in the past we always named our variables.  In JPR’s demo he only had object names.  No variable names.  In fact, calling objects by their variable name adds an additional layer to the call.  Internally, 4D maps the variable to the object and then call the object.

4D Write Pro

Continual evolution to have the same feature set of current 4D Write.  Written entirely in 4D.  No rulers yet.  No more ribbons but rather user / developer customizable palettes.  Configured by passing an object. Current 4D Write code will have to be re-written but existing documents can be imported into Write Pro. Nice interface.

QR and Label Editor

New versions written in 4D coming.  Needed for 64 bit compatibility.  Nice clean more modern interfaces.

Mirroring

Mirroring is critical for 24/7 databases.  Requires substantial plan and 2 people to oversee.  Unless you are available 24/7/365!
Won’t go into the details of the options for mirroring as they are multiple and need to be configured for each situation.

Milan stated that clearly that if your database does not have to be up 24/7 there is NO need to mirror. Just RAID your data file and then put the log file and 4D Backups on another disk.

4D Tags

4D Tags continues to evolve with the addition of 4D CODE.  Some of the original Tags are no longer as useful but kept for compatibility.
Keisuke Miyako demonstrated the ease (well easy for him!!!) with which you can create a template to export to Excel which include not only data but formulas.

SMTP_QuickSend

The latest version now accepts MIME preformed, not just plain text or HTML.  It is now in many cases more powerful (and easier to use) than building a full message with the complete set of SMTP commands. Keisuke demonstrated code on how to generate the MIME part.

GitHub

Keisuke has a wide variety of plugins.  Check them out.  No more needs to be said.

Miscellaneous Tips

Use Find in Field to check for uniqueness of a field based on a single field value. Use SET QUERY DESTINATION (Into Variable)  and then your QUERY for multiple field values.

Inter-Process variables are only visible to the Client not the Server. Semaphores are Server based.  Do not combine the two.  (I got burnt on the first one here.  I recently moved some code to be Executed on the Server that broke because it called a routine that included an inter-process variable that I define at Startup on the client.- i..e. directory character.)

A DELETE SELECTION command should be wrapped in a TRANSACTION.  It is basically a series of DELETE RECORD calls.

Other

The were many other interesting presentations.  More specific in nature.
Keisuke and Olivier are incredible resources. I asked Olivier a question about  a particular query and he promptly asked to review my database and pointed out some quick optimizations.
Many of the usual suspects were there, but there were a noticeable of younger programmers and new comers to 4D.  Pre-class had a jumpstart 4D class!
The whole 4D team and our rebel alliance (iNUG) are approachable, willing to share ideas.  That to me was the big benefit of going to the Summit.

Thanks to all.

Mitch

8. Apr: Kirk Brooks auf 4D iNug Technical

Another point where neo4D deviates from ‚traditional‘ 4D is the matter of
process variables. The clear message I heard was that large numbers of
process variables is no longer a good approach. The better solution is a
single, process level c-obj variable.

This is a major change for anyone who learned 4D on a version less than
v14. I was beginning to play with this idea before the Summit but had some
hesitation because it seemed too easy and I was worried there would be some
unforeseen performance hit. Turns out it is easy it’s what the cool kids in
the black shirts are doing too.

How’s it work? When you initiate a process initiate a process c-obj var.
‚prosObject‘ for example. I made a little method to do that and start off
with things like the method name, a list of parameters passed to it, which
process created it. That’s it.

I use process vars primarily to manage:
– process state info
– data used in more than one method
– data shared between the method and a form
– form objects

​** Process state
I display customers, vendors, orders, ​and so forth in separate processes
most often. These processes may or may not have the record loaded so there
has to be a process variable identifying what record is being used. I also
have some processes and stored methods that fire off and run until a
process var named ‚done‘ is true. I have some methods that start a new
process by filling a set of arrays then presenting them for user selection.

All these uses can be changed to reference the c-obj. In the first case
with:

OB SET(prosObject;“vendorId“;$1)

I can set the done flag with

OB SET(prosObject;“done“;False) and so on

Basically any time I need a process variable for something I can simply set
and get that value from the process object.

​** What about forms?
The message about forms I heard over and over is to embrace dynamic
variables for form objects and manage forms using object names. Especially
from JPR. This simply means to stop putting variable names in the form
objects you create and use meaningful object names.

To this end it would be nice if 4D tasked an engineer to spend a day or so
to update the widgets and forms automatically created by 4D to follow this
principle.

​
Talking with other devs I sensed a reluctance to really embrace this
approach. I was put off by it too at first but the pay off is worth the
effort. Forms become much more dynamic and easier to maintain. And when you
do need to share some data or setting between a method and a form object
you can use prosObject. Objects like buttons don’t need to have a process
variable associated with them. 20 years ago they did but now they don’t.

One source of a lot of my process variables are labels, messages, inputs
and text blocks. Like a label next to a field or variable that is changed
depending on the state of the record or the particular set of choices a
user made. Or messages created in a method that need to be displayed or
logged somewhere else. I have some specific entry dialogs for things like
time & date and more specific things I need a user to provide and then use
in a form or method. These are places I create a form and use a process var
to manage the interface with the user. Those forms can easily be configured
to read from and write to prosObject within the context of the form itself.

The list goes on but you get the idea.

** But what about performance?
For myself I had a nagging sense that this approach would result in some
sort of performance penalty. Perhaps because individual process vars
(vDate, vSpecificItem, vText1 … n) would take up less space and be
accessed faster than a single large c-obj. Or that referencing form objects
using by name was going to be less efficient than manipulating process
variables. The only people I found in agreement with this were other devs
like me. All the technical people with 4D I asked about this dismissed it.
JPR called out some of these things directly in at least one of his
sessions. I particularly remember him showing some code like this:

ARRAY TEXT($ar_TableList;0)
For ($iTable;1;Get last table number)

If (Is table number valid($iTable))

GET TABLE PROPERTIES($iTable;$flInvisible) If (Not($flInvisible))

APPEND TO ARRAY($ar_TableList;Table name($iTable)) End if

End if

End for

Nothing new here except it gets called a few hundred times in the course of
the form. If I were doing that I’d think it was ‚better‘ to call it once On
Load and save the results in a process array. JPR was emphatic on this
point – “ I do not care! It does not matter!“ he said about the time it
takes to run. This sort of thing is not the processing drain it once was.
Consequently the optimizations of years past are no longer valid now. He
acknowledged this is directly opposite his advice from 25 years ago.

In my own testing a while back I found the differences in referencing a
c-obj so small it would only matter in cases where I was running some tight
loop for hundreds of thousands of iterations.

The performance benefits of reducing the size of our process variable
table, however, are more immediate. I’ll preface this bit by saying I am by
no means an expert at this level of software engineering so speak up if I
get something wrong. In a compiled app every process loads the process
variable table. That means every single process variable is created and
initialized in every process. In 4D server that means there’s a copy on the
server side (the ‚twinned‘ process) and on the client side. This isn’t
necessarily a big deal, especially on the clients. On the server side it
can be if you have a large number of clients and/or web processes. Figure
your basic variable table is 6k (a number I heard used), and 50 clients
each running 20 processes. Then add some web processes and whatever stored
methods may be involved and you’ve got a chunk of memory used on the
server. The great majority of which is most likely unused because it’s
unlikely every process is using every variable. It’s more likely each
process is using a small subset. That table could be reduced to a couple of
dozen or fewer variable using c-objects.

Looking at my own databases the largest I have is a few hundred process
vars. If you have several thousand this could make a significant
difference.

** Other Advantages
During development you could know the state of all process data by reading,
perhaps logging, the single prosObject. In fact this would make it easy to
make a snapshot report of the state of every process for diagnostic
purposes.

Within a component you can reduce the chances of name conflicts by having a
prosObject with a different name for the component to use. This could also
be used to share data with the host.

You could ’store‘ and ‚restore‘ the state of a process, at least the user’s
choices for that process, from the c-obj which can be easily stored in a
c-obj field of simply stringified to text.

This same approach can be applied to interprocess variables as well.

** Interprocess communication
Variable to variable between processes will need it’s own variable I think.
I haven’t tried but I doubt you could use Variable to variable or Set
variable with OB SET to update an object in a different process.

You could incorporate a c-obj specifically for ip communication either by
using and actual ip var (<>ipProsObject) or with method scope
(prosObjectExternal) and use Set Variable or Variable to Variable with it.

** Don’t throw out your current work
The message was clear that this is the sort of direction we should be
moving toward in programming 4D. It is equally clear there is no pressing
reason to get involved in painful overhauls of existing code. Old, mature
projects are going to be fine the way they are. We all know these things
move slowly in 4D world. I see this myself when I notice a datestamp on
some of my work from 15 or 20 years ago. Would I do those things
differently now – usually. Is it worth the time to refactor the existing
code? Sometimes. Do I HAVE to refactor the existing code? Usually not.

For new projects though it’s a different story. Starting anything new is
the time to adopt the newer principles and techniques.

—
Kirk Brooks
San Francisco, CA
=======================

8. Apr: Tim Nevels auf 4D iNug Technical

Would trigger be running in pre-emptive core?

Short answer is yes.

Long answer is you have to choose on a per method basis. New method property: run preemptive, run cooperative, or „indifferent“. Default is indifferent which is basically cooperative.

If you want a methods to run preemptive you set it. As long as there are no UI commands like DIALOG or OPEN WINDOW it will run preemptive when compiled. Preemptive only happens when compiled and on 64 bit.

All methods called by this method will also run preemptive and also must not have UI commands. The compiler will check for this and do deep inspection to make sure any code path or call chain is cool. Otherwise you get a compile error.

So for your UI you still run cooperative and can access IP variables and run just like now. But if you want to so something that you want preemptive, have a method with the new setting turned on and make sure all methods it calls has no UI commands.

Indifferent setting is so that the method can be called from a preemptive or a cooperative method. It’s cool to run it both ways.

So since triggers don’t have UI commands you should be able to set them to preemptive. Not sure how you set that. Might be a new trigger property or database setting. Or you just have to call a preemptive method from your trigger.

All details of this has not been finalized yet. Expect to see it appear in v16 R releases.

Still very exciting and something we’ve wanted and needed for years.

Tim

8. Apr: Kirk Brooks auf 4D iNug Technical

In addition to the other information already posted I’ll add my interest in
the further implementation of c-object (JSON) into 4D. This was signaled by
the QUERY BY ATTRIBUTE command in v15 which allows you to query the
contents of a c-object field. This was sort of a big deal all on its own.
It means you can create a database in 4D that would mimic a Mongo (NoSql)
database, for instance.

I posted a link to an excellent article about SQL and noSQL last week.
NoSQL is used heavily in the world of web and social media apps. Why does
that matter to a 4D dev? For most existing devs it probably doesn’t. For
anyone currently not using 4D I think it means the platform has the
capability to easily interact with that world. And it’s a big world.

JPR and Olivier Deschanels did presentations showing why this sort of
capability is important and useful to straight up 4D devs as well. For one
thing a single c-object field can be a replacement for a subtable. Think
about phone numbers for a contact record. There may be zero or many
numbers. This generalizes well to a JSON and it can be searched on. Olivier
pulled out the old 4D airlines demo database (at least I think that’s what
it was) which had maybe 10 or so tables with numerous relations and
illustrated how that same data could be represented in a single table where
the records consisted of a mix of a few fields and some c-object fields.

JPR, in the post class, talked about the efficiency these new structures
offer. Going into the Summit I was reticent to try too much of this for
fear the performance was going to be slow compared to ’normal‘ 4D
approaches. This is not the case. In fact JPR stressed that to fully take
advantage of the preemptive capabilities, when they are available, your
code will need to be reworked using these newer approaches.

Miyako’s post class on SVG really sort of blew everyone’s hair back. His
sessions can do that. His manner and presentation is so polite and
comfortable and the material so dense and valuable. Personally I can follow
along with most of it but if I try to actually do some of the things as he
goes along realize I have to wait until I can work through it on my own.
Consequently at the end when he asks for questions I don’t have any because
I’m confident the answers are already in his materials I just need to read
them. I think I’m not alone in that thought.

His presentation on sending emails was another nugget like that. Fabulous
information I look forward to working through. Big take away for me is that
SMTP_quickSend (sp?) is actually more useful, in some cases, than the full
blown approach. I hadn’t looked at that command in years – now I see I need
too.

Balinder’s component for automating updates of 4D structures built on the
information Aichim Peschke presented. Aichim’s session discussed the tools
4D provides for doing this. Balinder’s session presented the component he’s
written that uses these tools to offer a complete system that allows you to
set up a network repository of version number and current structure file.
The component will automatically poll this repository for the current
version and if a new one exists ask the user if they would like to download
and install it. If the user agrees the download takes place, the files are
updated and the database restarts. He’s even included a neat feature for
providing release notes. This is pretty cool.

Olivier also did a sessions demonstrating using Leaflet js. It was another
example of the implementing the JSON capabilities baked into 4D now. Again,
very good. I’ll be putting up some more about this later on too. I like it.

There is more but I need to run now.

8. Apr: Kevin Abraham auf 4D iNug Technical

Synopsis from my point of view:

1) Pre-emptive core based processes in a future R release of v15 and in v16 from the get go. The price for this is that these Workers (that is term 4D was using for them) should not touch IP Vars for obvious reasons. IP vars are not thread safe and to share them between cooperative processes and possibly multiple pre-emptive processes would not be safe without semaphoring every instance where you access them which would defeat the purpose of having pre-emptive processes in the first place. Instead other ways to communicate back from the Pre-emptive has been provided and I think the best ways to think of the pre-emptives would be as headless (will get to that next) daemons. Also, pre-emptive processes should not display anything on screen, instead there is a mechanism provided to report progress on whatever they have been asked to do back to cooperative process for screen display. This is very powerful but will cause a lot of bugs if people are not VERY careful.

2) New 4D Write Pro (same license as current write) which will support 64 bit only.

3) New 4D View Pro (same license as current write) which will support 64 bit only.

4) Wakanda and 4D Professional Services now being offered. Contact your 4D sales rep.
(Insert other stuff I did not care about here… Hopefully someone else will report on some other stuff.)

7. Apr: Q&A und Promo-Video

https://m.youtube.com/watch?v=4mxL6hCo8Dk

4.-7. April findet in Portland die erste der drei 4DSummits 2016 statt. Um Neues zu erfahren Wayne Stewart auf Twitter folgen oder ab+an hier nachsehen.

2. Tag

Q&A

Q: Preemptive multi-threading, can you have read only access to IP variables? (e.g. as a constant).

LE: This is something that we want to provide you with. Get Process Variable does not work currently but will work but slow.

Q: Faceless server on linux, will it happen?

LR: No definite answer!

Q: What is biggest threat to 4D in the next 18 moths?

ET: Nothing is anticipated as being a problem.

LR: Apple is moving to common apis between iOS and MacOS but with move to cocoa 4D is not really worried about this.

LE: It might be hard to get 4D to run in an Apple Car

Q: Touch event support in 4D?

LE: Click events work well, some thought being given to this

LR: Using more native APIs which will bring in these sort of things.

Q: Management of client sessions in coop thread, will this be moved to preemptive, to allow more clients?

LR: Yes, that is the idea.

Q: Version control for 4D?

LE & LR: That is a good question!

LE: Not there but it is something that 4D is looking at internally

Q: What is 4D doing to spread the word about how wonderful 4D is?

ET: In France they have partnered with a university to run a course, they would like to do something similar in USA.

BY: New website (coming soon) will be friendlier for new people.

4D Promo video from Wayne Stewart on Vimeo.

In summary, v16 is all about scalability!

Choose which tables have priority

Graphs moving from 7 properties to 30+

4DVP variable row height + build complex form entry items.

4D View Pro, cocoa, 64-bit ready. 4 main requirements, display records, spreadsheet, report & printing and calendar, gantt charts

1. Tag

Each server can be preemptive

• New architecture for deployments, remove prefs from 4D app bundle. Easier to deploy and have read only apps.
• Multi-threading coming to platform, pre-emptive in 4D code. v15r5 release & v16.
• 64 bit everywhere, 32 bit will sooner or later be retired. They will be available as long as needed.
• Professional services team created by 4D to work with 4D developers, advice, co-develop, training