Chant

It might be better to split into two topics, nx500 and nx1. Or front end and back end. The things Otto is doing to the nx500 are amazing, but less similar to what I am doing with going into the firmware. It may be better to have things split up so updates dont get lost in the posts. My work has less shareability and its not something someone would want to play with unless they want a nice nx1 sized brick. I will be doing a comparison on the nx500-nx1 firmware to see if things can be added or removed by just copying the .cpp files.

But if you compare the fcc internal photos https://fccid.io/document.php?id=2507770 nx500 and https://fccid.io/document.php?id=2394073 nx1 you can see the change in size of the drime-v and drime-vs. There is also the unreleased nxmini2 which had the drime-vs in it also https://fccid.io/document.php?id=2594973 nxf2

From my end the later updates of the firmware do have a lot of changes, obviously due to file size less changes, but the specific documentation is there. But things are still progressing on my end.

38 minutes ago, tugela said:

More likely a limitation of the processor, not the sensor. The sensor does full reads at 240 fps.

Im finding references of sensor read out in the fimware. But without full data Im not sure if things can be changed. If this sensor is similar to the S5KVB2 or a different roll out. Having the data sheet would shed some light. The cmosis cmv12000 and cmv20000 are both global and rolling and its implemented in code which readout it does. Unfortunately the S5KVB2 data sheet is not something that is attainable and if it was Im sure it would be under nda. But Its all things to test once I have it all sorted. My notes are filling up pages.

31 minutes ago, Pavel Mašek said:

WOW, I have just received email from SAMSUNG to download link to NX1 SDK!

(well I am quite scared to upload it somewhere, but I think I will do it). I will try to download it in the evening and let you know.

Chant, is it the one you have? Do you think it will be useful? I am not any IT expert, so I really do not know...

 

Dear Customer

Thank you for requesting an NX1 SDK.

 

Unauthorized distribution of the SDK may be unlawful.

You are not authorized to upload, copy, or redistribute the SDK in any way.

 

Click the link below. After input your E-mail address which you used for SDK request,

you can download SDK.

Contents:

• Sample Source: Sample source codes are provided for your reference.

                      (Tool : Visual Studio 2010 recommended.)

• SDK User Manual: This document contains general information of SDK API and user instructions for the SDK.

 

Thank you.

Sent you a message regarding this

11 hours ago, Pavel Mašek said:

Chant, could you please share the SDK? I think we could add it to github ( https://github.com/ottokiksmaler/nx500 ). One place to show actual progress and results of NX1 hacking would be nice (SD sripts hacks/ firmware hacking).

Its not the sdk for tizen sorry to say, but the sdk for the srp or so it seems to be. I havnt tested anything yet and its not something that would be of use to anyone unless they are at the point I am. If I do come across the tizen sdk I will post it for sure. The version the camera runs is older than the newest version, I have yet to have a deep look at what they have online, to see if it contains resources for the camera that are just not enabled though. As they dont do much house cleaning code wise. who knows what they left behind. Also the major update of firmware 1.23 might have some internal tizen info but I havnt started on analyzing everything yet so Im not sure what they did.

The code excerpts I posted are from the firmware. The statement about the camera being able to do 240fps is something they didnt remove. Also found a bunch of tizen watch stuff. Bad housekeeping is good news.

But I have yet to find a video output path that is concise. Tons of data on the hevc and everything else, but its what pointed me to the srp and needing to do more research. There way a camera would dump data to an onboard hevc is pretty well documented. Follow the mipi csi-2 bridge a few steps to get to the encoder. but the difference is, if it was indeed just a "hard" core hevc in the drime-v it would be more self contained. Taking what I know of chip construction and "hard" core asic  implementation. The more I do the more I find on how it works and what can be done.

The samsung srp research is also to see if thats what they used, but it it was an onboard hevc asic there would be a lot less code for what Im finding if you go by what tends to control hardware. With luck its srp because Ive found examples of it beings used to do a full 8k6p stream with a very small srp block and the drime-v is a very different beast from the exynos.

But ill just leave these excerpts here.

"eSensorModeRollingStillModeOff
eSensorModeRollingStillModeOn
eSensorModePanoramaFull"

"30CBayerImageInformationLiveview
34CBayerImageInformationLiveviewWide
36CBayerImageInformationLiveview120fps
36CBayerImageInformationLiveview240fps"

"LVIEW_PP_CORE_UD
LVIEW_PP_CORE_UD_24FPS
LVIEW_PP_CORE_120FPS
LVIEW_PP_CORE_240FPS"

"eFrameRate24fps
eFrameRate25fps
eFrameRate30fps
eFrameRate33_3fps
eFrameRate40fps
eFrameRate50fps
eFrameRate60fps
eFrameRate100fps
eFrameRate120fps
eFrameRate200fps
eFrameRate240fps"

"binningoff
HVbinningon
Hbinningon
Vbinningon"

There has been a few stating that they would like to tip/donate, which would of course be muchhelp,, as I seem to be doing this more than my 9-5 haha! But it is not at a quantifiable stage at the moment, What I have found out is pretty much just a key, and if the key can open the lock is not yet determined. Even with the sdk things of this nature are not 100% 

At any step of the way if they did something undocumented it can halt progress to a point the modification cant be done. Ive been lucky so far with things is all!

But if knowing that what Im doing may not bring a firmware mod to light people still wish to donate then I can put something together. But I started this not knowing there was anyone wanting this camera modded, and I would still do this if no one wanted to tip/donate.

That being said the next stages of what I am doing are copious research. There are white paper behind paywalls but my local university seems to have most of the books on file so I will be acquiring those to get a better idea of the way samsung chose to code.

Personally I am also interested in the lens mount for a working adapter to other lens systems. So the control files are something I am also looking into. As well as if hevc is the best choice for encoding or if something like the redcoderaw would be more suitable which is just their modded version of mjpeg2000. dct vs wavelet is something interesting to read about. 

Just now, sidi said:

I agree as long as the recording limit is removed.

That is something that would be very simple to remove compared to what I just explained. A secondary task though, first we get to the good bits, and everything else is second. The way the srp even loads in is something that isnt well documented, let alone on a camera.

Thanks to some ideas I had regarding hardware and if the nx1 was a true uhs2 camera or just backwards compatibility with uhs2 and thus only being a uhs1 camera I have found proof.

page 3 main pcb shows the full sd slot.
https://fccid.io/pdf.php?id=2394050

The sd slot http://www.alps.com/prod/info/E/HTML/Connector/SDMemoryCard/SCDA/SCDACA0200.html
 Made for uhs2. So the reality of the full speed uhs2 sd bus is possible.

Talking to a few people I know who do front end. Seeing what can be done about having a set up like magic lantern for ease of use. That or writing the menu itself to add in the options. If its a possibility we will try to keep is as far from sonys menu as possible. If anyone has used their gear.

40 minutes ago, Syme said:

According to this Anandtech article the Samsung uses something they call the SRP (Samsung Re-configurable Processor) in the DRIMe-5 SoC. They call it a CGRA (Coarse-Grained Re-configurable Architecture), which probably means it is less flexible than a FPGA but more flexible than traditional IP blocks. I have no idea what their source is, but Anandtech is known for being very reliable. Details about the SRP online are very sparse, but there are a few research papers about it being used as an experimental GPU. Apparently it is used in the last generation of Exynos SoCs as an audio processor (not for their latest chips, though).

The SRP is definitely not being used as the GPU in the DRIMe-5, since the kernel source shows it uses a conventional GPU just like in the Exynos SoC it is based on. It also isn't being used for HEVC encoding, since Samsung has special-purpose hardware blocks to do that. As to what they are using it for, I have no idea at this point.

This is good news to me, it has answered a lot of the questions on what I have been finding in the back end. It is a dsp block, from what I have found so far there is some documentation out there. Lots to read tonight. But this has made a huge step forward for me. I dont know how I never came across the article mentioning that information before I found the fpga+asic article I found!
But much thanks!

17 hours ago, Syme said:

Out of curiosity, how did you find the clock speed?

It was in a cpu config file. But in early firmware, I am plotting the changes and original values to see what sort of leverage can happen with them.

12 hours ago, derderimmermuedeist said:

If i see it right (in the Chart) then the maximum Bitrate for the H.265 HEVC is at Level 5.1 (found this Level in the File-Infos of my NX1-Videos) 160.000 kbit/s, approximately 156 Mbit/s.

And interessting too, the maximum Frame rates:

c4k @ 60p

UHD @ 64p

an FHD @ 256p

This is something that I have also been looking into. With luck it is just the change of an output flag and see if things can be pushed to a higher limit. HEVC and other compression systems are less intensive with less compression, so in theory it should be able to bump up to the bus limit and have less compressed files. In the later firmware the change from 15fps to 25fps for the raw output sort of shows the ceiling of bandwidth the processor can handle. Give or take 30MB raw file x 25fps data stream is 750MBs/6000Mbs

My goal is to see how close to the spec of a red weapon the nx1 can be. HEVC is sort of a pain but the way it works is more like the wavelet based .r3d files when HEVC is at lower compression rates. But the more I find the more interesting things get. And the things they did for this camera make it vastly unique compared to a canon or a sony.

5 hours ago, Marco Tecno said:

I guess the ideal thing would be to write a virtual machine for running experimental NX1 fw into it. It won't guarantee that no risk is taken once ported onto the real thing, but it would actually lower the risk by a large margin. I don't know if this is easily feasible, though.

 

Another great (but difficult thing) thing would be a "fw pushing" service from a PC, so that you can push a new fw onto NX1 no matter it's state.

Another ideal thing would be a recovery, to be invoked by some keypress.

The complexity of the nx1 makes that quite hard, if I am correct and there is an fpga/cpld core on board it gets more tricky. I have seen some signs though. No confirmation yet. More work will be done tonight on the dive though. I have compiled a large data sheet already on the things I have found in the first firmware update. But going though the code takes time.

And if people are having issues with HEVC I highly recommend getting a GTX960 so far the only gpu with fully enabled on board HEVC encode/decode and boy does it work well. And they are not very expensive. If you go by how HEVC does compression, when you transcode it would be in the Gbs if the data rate is doubled on camera

The list of additions are interesting, I have to admit the focus will be on video and all other aspects will be secondary.

Found a few things. DRIMe-V clock rate seems to be 200mhz. And that seems.. low. Im expecting an NX1 in the next few weeks, so that is one thing I will test. May have to do some heat dissipation help, but from other things I have found in the fw that should be able to be pushed. If things are on the level of the sony a7(s) line up with their over heating issues I think a tad below that will suffice. The upper limit should be 700mhz+ But erring on the safe side will always be more logical.

**Edit** In additon to that finding, I will have to track the changes to see if they upped the clock rate in the later fw. I am working in the first revision as that has the largest file size, so I assumed it would have the most debug changes from the 1.0 layout, with the other firmwares focusing more on updates and camera usability changes.**

I wasnt intending to work on this tonight as I have some graphic design work I need to do, but the more I look, the juicer things get!

When it comes to that point things can be figured out. As I stated, I started this project for my own ideal goals for this camera, I guess its just a plus that more people are interested in expanding what the nx1 can do. The next few weeks will dictate what is possible back end wise. What otto k has also done would be useful. As building a front end like ml would be more user friendly compared to what was built for the gh2 and ptool, and that sort of firmware update system. 

If we could get a list going of what people would like to see it could help focus the work better. Unlike other cameras the firmware is complicated but not near as obfuscated and vastly lacking encryption as other systems. Open source really makes it easy even if the sdk isnt public.

Depending on what I find there could be an option for a 1:1.29 lossless or a higher compression visually lossless hevc. Creating a custom lut for a more red like log seems possible also.

 The firmware seems more of a proof of concept that they left it open to change so many things. And more so to show they can go head to head with the established makers and make something better. Since this thread has garnered more interest I have shifted to put more hours into this dive I am doing.

Well afaik the NX1 and the NX500 have different versions of the DRIMe V (NX1) DRIMe V-S(nx500) But I have the firmware of the nx500 which is of interest for me also, again cheap b camera and see what it can do and how much it can be pushed and sacrifice the length of life. But that is secondary to the NX1 for me. I can browse the FW in the next few weeks and see if things are vastly different though. See if the camera control files are written similarly/

I started this research when the nx1 first came out, saved and tracked the firmware, and then started digging into when the rumors of the exit started happening. The tech inside this camera is just magic. Compared to what other companies are doing its a few years ahead. And due to samsung being able to roll their own chips it seems they didnt cripple anything because it was too costly. Im not going anywhere! 

47 minutes ago, MountneerMan said:

Someone should try and recruit some of the ML community development staff.

I am sure there are a bunch of really smart programmers who are starting to get board of working with Cannon cameras and wouldn’t mind getting in on the ground floor of a new development.

From what I am hearing it sounds like making software for Samsung cameras is going to be significantly easier than cannon as well.

Do you think this is a limitation of the hardware or could this be unlocked with software?

It was written in the .cpp files showing the hz of the sd bus, this would be an incremental increase test that would just be raising it until it crashes or to the limit stated by the sd association. And if it changes depending on what type of file is being written to it. As well as tracking the sensor and chip heat out put to see if it is underclocked for a reason. Lots of fun.
 

1 hour ago, kidzrevil said:

Sounds good. Can you produce any evidence that you are able to hack with the custom firmware ? A "hello world" would be enough to start funding the project

Not at the point yet, still tearing it down. The base tizen didnt seem to change much through the updates. It was the control files for the camera, written in c and a whole new ball game. I do about 15 hours of work into this a week, which isnt much considering but things are moving. If you wanted me to just make hello world appear it wouldnt really prove much, at least not in the way that this camera works.

 

1 hour ago, Marco Tecno said:

chant. very interesting. a big question: do you have a vr emulating a nx1? did you write it?

Have not got to that point yet, still diving into the code and doing a control file comparison, as the updates for the camera happened they state what they changed, and that is what the major key point of what Im doing is. Its one of the nexts steps though. 

I can update as I progress and post summaries of what I find. Just to find the rgb888 and lut info I had to look though about 15000 lines of code. Its tedious. But worth while for me in the end, so I will continue to dig and deconstruct. And update!

New to the forum, but I have been working on the opening of the firmware for quite a few months. The second I heard about this camera coming from a modded gh2 I knew things were going to be fun.

I had plans to release what I found after having a stable custom build.

Good news after reading this thread as I seem to have quite a bit more discovered than most.

-bootloader

-3d lut info

-rgb888 info

And quite a few more things that make this an ideal camera to mod.

In the interview they basically implied they either had asics or fpgas built along side the arm cores.

"DE: When we were talking previously, you said that the architecture of DRIMe V is very different than DRIMe IV. What sorts of changes are made between them?

JK: The biggest change is the structure of the Image Signal Processor. The DRIMe V ISP is very different. Most ISPs have key parts of the processing hardwired to get the needed speed. What's really new with the DRIMe V is that the "hardwiring" can be reconfigured.

DE: That sounds very significant, although I have to admit I don't know how common it is that ISPs are actually hardwired.

JK: There's not much variability in them. Sometimes you'll get thresholding changes and small numeric variables, but the image path is usually very locked down...

DE: It's pretty much, a fixed pipeline.

JK: With a few tuning pieces, yeah.

DE: Whereas this is more configurable.

JK: Yes.

DE: And it's configurable not at the level of changing a firmware program, but you're actually changing the hardwiring. There are switches you can use to change the configuration of the functional units?

JK: Yeah, basically. And Samsung's software people are all over it; they're pretty sharp and I believe that we're really going to be able to tap fully into the DRIMe V's capabilities for the NX1.

DE: So even though it's hardwired, some of the connectivity is programmable through the firmware, so we can actually look for continued "firmware" development that would continue to improve performance, including the parts that are implemented in hardware.

JK: Yes.

DE: So there may be significant improvements in the near future. That's pretty interesting, because it sounds like it's already starting out pretty capable.

JK: And not just image quality, there's a lot going on in terms of programmability, like we talked about Samsung Auto-Shot mode before. Here's the UI for it."

TLDR: fpga or cpld or asic that can be reconfigd.

Now a few years ago samsung released info about adding fpgas into asics http://www.eetimes.com/document.asp?doc_id=1200317
 

And have teamed up with Lattice on a few projects http://www.oregonlive.com/silicon-forest/index.ssf/2014/04/which_oregon_company_has_a_chi.html

The back end camera control files seem to be written in .cpp so diving deep into the firmware to do a comparison of hex/string/hash changes and tracking the changes across the updates is what I am doing right now. And seeing which values can be modded. Recompiling is a different story but what it seems to be is that building a smiliar man in the middle to what magic lantern would be possible. Have a gui that would simialr to ptools in camera using the web browser capabilities.

Now this is not my full time work so Ive been progressing slowly as I have many projects on the go.
This is also coming from someone who doesnt yet have an nx1. Doing the proof of concept and running the fw in a vm to emulate the possible bricking before things are bought.

I am also tracking similarities to the nx500 to see what things are possible in that camera. Nice b camera which should be able to compete with a shortening of its life but at the cost 10x performance is worth the lifespan limit imo.